Hey, everyone.  The list has been quiet for a while, so I wanted to see
what everyone was up to these days.  Hopefully no one's lost interest in
the whole open access to government information thing. :)

Actually I haven't done much GovTrack-related stuff this summer.  But
has anyone made any headway in state-level data?  New ideas?  New
website finds?

--
- Joshua Tauberer

http://taubz.for.net

** Nothing Unreal Exists **

On Aug 30, 2005, at 8:04 PM, Joshua Tauberer / GovTrack.us wrote:
> Actually I haven't done much GovTrack-related stuff this summer.  But
> has anyone made any headway in state-level data?  New ideas?  New
> website finds?

I struck out finding the current terms for Illinois reps and
senators, and stopped there.

//Ed

On 8/30/05, Joshua Tauberer / GovTrack.us <tauberer@...> wrote:
>     Hey, everyone.  The list has been quiet for a while, so I wanted to see
>  what everyone was up to these days.  Hopefully no one's lost interest in
>  the whole open access to government information thing. :)

I haven't lost interest.

>  Actually I haven't done much GovTrack-related stuff this summer.  But
>  has anyone made any headway in state-level data?  New ideas?  New
>  website finds?

I haven't done anything useful.  It's a matter of time.

--- Joshua Tauberer / GovTrack.us wrote:

> Actually I haven't done much GovTrack-related
> stuff this summer.

Me either, the beautiful summer days/evenings seem to
keep calling me outside.

> has anyone made any headway in state-level data?

Last I left things here in Cali I had completed
xmlizing the CA Statement of the Vote going back six
years[1]. I was still working on merging this data
into a master politician list w/ unique ids.

--- Edward Summers <ehs@...> wrote:

> I struck out finding the current terms for Illinois
> reps and senators, and stopped there.

I think perhaps the most complete way to get the terms
served (and term durations) is to derive it (through
election results).

Scott
-----
[1] http://www.pdgov.org/data/US/CA/election/



____________________________________________________
Start your day with Yahoo! - make it your home page
http://www.yahoo.com/r/hs

On Sep 1, 2005, at 11:19 AM, Scott Beardsley wrote:
> I think perhaps the most complete way to get the terms
> served (and term durations) is to derive it (through
> election results).

Hey, that's a good idea :-) I'll see if this gets me a bit further.
At the moment I've got some scrapers and rdf generators [1] but
nothing very useful apart the names and addresses are getting parsed
for senators and representatives.

I tried contacting the folks at ilga.gov who make the Illinois data
available via the web, but haven't gotten  a response. Would be nice
to work directly with them rather than scraping. Any ideas for
interjecting ourselves into these organizations?

//Ed

[1] http://www.inkdroid.org:7000/textualize/browser/govtrack-il/trunk/

Hello,

I'm still around.  I haven't done much the past couple months for work
related reasons.  I have just finished moving to another place of employment
and a long needed vacation.  I had made some headway with Virginia Senators
and Congressmen.  I have most their information imported into a Postgres db
with the exception of dates for some of them.  For some, the information is
quite specific as to election dates, however with others it's vague, but
I'll make it work.  I should be able to get moving again in a couple weeks
while my wife is on vacation with our son.

Ryan

--- Edward Summers <ehs@...> wrote:
> Any ideas for
> interjecting ourselves into these organizations?

I'd say contact the government agency responsible for
putting out the data. In Cali it's the Secretary of
State but in Illinois it looks like the Board of
Elections[1] is responsible.

Illinois looks like they are a few steps ahead of
California as far as uniform formatting. I did a
little surfing and you can get 7 or so years of
candidate[2] and election info[3] (in multiple
formats). Put the two together and you can cross
reference the names/positions in legislation[4].
They've even got a dtd for the bill status[5].

It's a round-a-bout way of getting everything we need
but I think it's the best approach. Mine data from the
sources responsible for creating/maintaining that
data.

What'dya think?

Scott
-----
[1] http://www.elections.state.il.us
[2]
http://www.elections.state.il.us/ElectionInformation/CandFiling.aspx?NavLink=1
[3]
http://www.elections.state.il.us/ElectionInformation/DownloadVoteTotals.aspx
[4] ftp://www.ilga.gov/legislation/
[5] ftp://www.ilga.gov/Resources/billstatus.dtd





______________________________________________________
Click here to donate to the Hurricane Katrina relief effort.
http://store.yahoo.com/redcross-donate3/

Hi All,

Please pardon a post that's a bit off topic for this list, but I thought
some might be interested because of the overlap with scraping and data
interchange formats.

There's a project going on to scrape the dozens of websites set up that
list people know to be "safe" after the hurricane. The goal is to get
that info into a central store available to web users for searching.

http://katrina.internet2.edu/wiki/index.php/Katrina_PeopleFinder_Project

An interchange format has been created:

http://zesty.ca/pfif/

If you are interested in helping, follow the instructions on the project
website.

Steve

Steve Andersen
ONE/Northwest, Program Manager
Phone: 206-286-1235 x17
Skype: Steve_Andersen

I've been fooling around w/ RDF a bit and I'm hitting
a wall. Maybe others on the list can help. I'm
gathering data (facts) from different sources.
Sometimes these facts may agree and overlap (ie last
name and first name) and sometimes they may contradict
each other (ie current address vs previous address).

What I would really like to do assign a source to each
fact. For example:

"John Kerry is a Democrat" vs
"According to the CA SoS, John Kerry is a Democrat."

in RDF I'd format it as:

<John Kerry> <is a> <Democrat> vs
<John Kerry> <is a> <Democrat> <CA SoS>

Why would I want to add the last bit of data? Later I
could assign reliability to each source giving one
more preference than the other. If I just throw out
the source information (for each fact) then we'd have
to go through this whole parsing process again. I
can't be the only one that wants this... any ideas on
a workaround using existing methods? I could add a
fact saying the source is such-and-such for fact#n but
that would effectively double the number of facts (and
I'd have to identify each fact somehow).

Really what I want is a way to maintain an ongoing
bibliography record for every fact I find.

Scott




______________________________________________________
Click here to donate to the Hurricane Katrina relief effort.
http://store.yahoo.com/redcross-donate3/

Scott Beardsley wrote:
> "John Kerry is a Democrat" vs
> "According to the CA SoS, John Kerry is a Democrat."
>
> in RDF I'd format it as:
>
> <John Kerry> <is a> <Democrat> vs
> <John Kerry> <is a> <Democrat> <CA SoS>

Hey, Scott.

The closest thing, as far as I know, is formulas in N3.  See the
Formulae section in http://www.w3.org/DesignIssues/Notation3.html.

With that, you could rewrite what you had above as:

{ <John Kerry> <is a> <Democrat> .
       <Bush>    <is a> <Republican> .
     ...
    } ex:assertedBy <CA Sos> .

I'm not sure how good an idea it is to use that, though, since I don't
know how those are handled by existing RDF tools.  Redland doesn't
support it (as far as I can tell).

My own C# library should handle it ok :)
(http://taubz.for.net/code/semweb).  In that, I'm doing basically what
you wrote: statements are quads rather than triples, with the last field
holding some sort of meta node.

Maybe we should add a page to the GovTrack wiki with things we need from
the RDF world.

--
- Joshua Tauberer

http://taubz.for.net

** Nothing Unreal Exists **

On Sep 7, 2005, at 7:52 AM, Joshua Tauberer / GovTrack.us wrote:
> I'm not sure how good an idea it is to use that, though, since I don't
> know how those are handled by existing RDF tools.  Redland doesn't
> support it (as far as I can tell).

In case you are looking there are a few utilities listed in the
implementations section at:

      http://www.w3.org/DesignIssues/Notation3.html

> Maybe we should add a page to the GovTrack wiki with things we need
> from
> the RDF world.

Great idea.

//Ed

Hey, all.

I thought I'd let you all know of some updates.

First, I've update the design of GovTrack a bit.  Nicer colors.  Some
pages have been made more understandable too.

I added a new Find Your Reps page using Google Maps to show
congressional district boundaries.  That's linked from the main page
("Who represents you?").  That was quite a challenge.  District
boundaries from the census have too many data points to render without
doing some pre-processing first, especially when showing all of the
districts in a state.  And putting little balloons in each district,
when the shape of a district can be quite wild, needed another trick.
But now that it's done, it's pretty neat.

I also added a new voting records page which compiles a table of voting
records based on the people and subject terms that you've already
selected to monitor.  It's a bit cumbersome now to get voting records
for people and subjects you haven't monitored, so I hope to improve that
in the future.

The last change isn't about the website but relates to our discussions
way back on RDF and the semantic web.  I updated and expanded my "What
is RDF" article that I first posted back in March.  Anyone new on the
list interested in proliferating open access to information should take
a look at that:
    http://taubz.for.net/code/semweb/whatisrdf/

Take care.

--
- Joshua Tauberer

http://taubz.for.net

** Nothing Unreal Exists **

Josh - Great work as usual.

Re Who-Reps-You: Yes, tough to map NYC congressional
districts.
One suggestion: you could link to the USPS Zip+4
finder. This would be esp. helpful as part of the
"The ZIP code you entered is within multiple
congressional districts" message.

Thanks for the update,
Peggy



--- "Joshua Tauberer / GovTrack.us"
<tauberer@...> wrote:

> Hey, all.
>
> I thought I'd let you all know of some updates.
>
> First, I've update the design of GovTrack a bit.
> Nicer colors.  Some
> pages have been made more understandable too.
>
> I added a new Find Your Reps page using Google Maps
> to show
> congressional district boundaries.  That's linked
> from the main page
> ("Who represents you?").  That was quite a
> challenge.  District
> boundaries from the census have too many data points
> to render without
> doing some pre-processing first, especially when
> showing all of the
> districts in a state.  And putting little balloons
> in each district,
> when the shape of a district can be quite wild,
> needed another trick.
> But now that it's done, it's pretty neat.
>
> I also added a new voting records page which
> compiles a table of voting
> records based on the people and subject terms that
> you've already
> selected to monitor.  It's a bit cumbersome now to
> get voting records
> for people and subjects you haven't monitored, so I
> hope to improve that
> in the future.
>
> The last change isn't about the website but relates
> to our discussions
> way back on RDF and the semantic web.  I updated and
> expanded my "What
> is RDF" article that I first posted back in March.
> Anyone new on the
> list interested in proliferating open access to
> information should take
> a look at that:
>    http://taubz.for.net/code/semweb/whatisrdf/
>
> Take care.
>
> --
> - Joshua Tauberer
>
> http://taubz.for.net
>
> ** Nothing Unreal Exists **
>
>
>

Peggy Garvin wrote:
> Re Who-Reps-You: Yes, tough to map NYC congressional
> districts.

Yeah, I know.  I'm not sure how to improve that, though.

> One suggestion: you could link to the USPS Zip+4
> finder. This would be esp. helpful as part of the
> "The ZIP code you entered is within multiple
> congressional districts" message.

Good point.  Thanks. :)

So what happened with Thomas's update, do you know?  I haven't noticed
any changes.

--
- Joshua Tauberer

http://taubz.for.net

** Nothing Unreal Exists **

The THOMAS "facelift" is in limbo. Technically done,
but not approved. I do not have a sense for when or
whether it will go through.

--- "Joshua Tauberer / GovTrack.us"
<tauberer@...> wrote:

> Peggy Garvin wrote:
> > Re Who-Reps-You: Yes, tough to map NYC
> congressional
> > districts.
>
> Yeah, I know.  I'm not sure how to improve that,
> though.
>
> > One suggestion: you could link to the USPS Zip+4
> > finder. This would be esp. helpful as part of the
> > "The ZIP code you entered is within multiple
> > congressional districts" message.
>
> Good point.  Thanks. :)
>
> So what happened with Thomas's update, do you know?
> I haven't noticed
> any changes.
>
> --
> - Joshua Tauberer
>
> http://taubz.for.net
>
> ** Nothing Unreal Exists **
>
>
>

While it's not really relevant for current government information I
thought it could be of interest:

"The Government Printing Office is setting an aggressive timetable
for its massive project to digitize nearly every federal document
published since the birth of the nation. "

http://appserv.gcn.com/cgi-bin/udt/im.display.printable?
client.id=gcndaily2&story.id=37227

Edward Summers wrote:
   > "The Government Printing Office is setting an aggressive timetable
> for its massive project to digitize nearly every federal document
> published since the birth of the nation. "

Very neat.

--
- Joshua Tauberer

http://taubz.for.net

** Nothing Unreal Exists **

Hello,

Congrats on all the work this group is doing. GovTrack has impressed
me quite a bit. At Joshua's suggestion, I'm posting here to see
if anyone has interest in working with my team on a related project
we're beginning.

My team makes campaign software for activist organizations and
progressive candidates, called Campaign Engine. We're proud of the
comprehensiveness of our product, which is also very easy to use. It
is LAMP-based, but it is a commercial product. You can read about it
here: http://www.mediamezcla.com/products/campaign/

For our next version, we want to implement a "find my legislator"
feature -- ideally that works on local, state, and federal levels for
50 states.

We're looking for coders who can use TigerLine census data to create
matching tables between ZIP+4 and local, state, and federal
legislative districts. (Or, if you recommend a different approach,
we'd consider it.) This is a for-pay, contract assignment.

Anyone who's interested should email me at bschaffer@....
Please include your resume, rates, and other relevant work information.

Thanks again for the work you're doing.

Best,
Ben Schaffer
Media Mezcla LLC

Hey, guys.

Today I had an article published on xml.com about bringing government
data onto the semantic web.  It should be the first in a series of
similar articles.

http://www.xml.com/pub/a/2006/02/08/govtrack-us-public-data-semantic-web.html

Hope not everyone has lost interest in opening up access to the government!

--
- Joshua Tauberer

http://taubz.for.net

** Nothing Unreal Exists **

--- "Joshua Tauberer / GovTrack.us"
<tauberer@...> wrote:
Not at all.
--Peggy

>
> Hope not everyone has lost interest in opening up
> access to the government!
>
> --
> - Joshua Tauberer
>
> http://taubz.for.net
>
> ** Nothing Unreal Exists **
>
>
>

On Feb 9, 2006, at 9:13 PM, Joshua Tauberer / GovTrack.us wrote:
> http://www.xml.com/pub/a/2006/02/08/govtrack-us-public-data-
> semantic-web.html

Nice work Joshua!

> Hope not everyone has lost interest in opening up access to the
> government!

The interest is still there, but my personal motivation for screen
scraping data from local Illinois data has waned due to other
projects. Thanks for the kick in the pants though. You are doing
important work.

//Ed

Hello,

I was looking at the raw XML data available on the GovTrack.us site,
and I couldn't find XML versions of the debates from the Congressional
Record.

Are these available?  Where would they be located?

I've been trying to build a parser for that data for some time, and it
has proved to be rather difficult.

Thanks,

-Bryan

alphasigbryan wrote:
> I was looking at the raw XML data available on the GovTrack.us site,
> and I couldn't find XML versions of the debates from the
> Congressional Record. Are these available?  Where would they be
> located?

Hi, Bryan.  Yep, they're here:

http://www.govtrack.us/data/us/109/cr/

> I've been trying to build a parser for that data for some time, and
> it has proved to be rather difficult.

Yeah, that was for sure the oddest text to parse, and I find bugs with
the parser pretty often.

--
- Joshua Tauberer

http://taubz.for.net

** Nothing Unreal Exists **

Bryan Helmkamp wrote:
> Actually, open doing some more searching, I found what I was looking for
> in the "cr" directories.  It appears that the newest data is a couple
> weeks old though.  How often does GovTrack update?  Are the updates
> found in the /data/ directory immediately?

Uh, yeah, apparently I forgot to comment something out after making some
changes and so GovTrack hasn't been fetching them for a few weeks.  Doh.

They're downloading now (and appear immediately in that directory;
updates are daily, when I don't mess things up).

Thanks for pointing this out!

--
- Joshua Tauberer

http://taubz.for.net

"Unfortunately, we're having this discussion. It's too bad,
because guess who listens to the discussion: the enemy."

Joshua,

Have you ever thought about opening the parser source code for collaboration?

-Bryan

On 2/20/06, Joshua Tauberer / GovTrack.us <tauberer@...> wrote:
> Bryan Helmkamp wrote:
> > Actually, open doing some more searching, I found what I was looking for
> > in the "cr" directories.  It appears that the newest data is a couple
> > weeks old though.  How often does GovTrack update?  Are the updates
> > found in the /data/ directory immediately?
>
> Uh, yeah, apparently I forgot to comment something out after making some
> changes and so GovTrack hasn't been fetching them for a few weeks.  Doh.
>
> They're downloading now (and appear immediately in that directory;
> updates are daily, when I don't mess things up).
>
> Thanks for pointing this out!

Bryan Helmkamp wrote:
> Have you ever thought about opening the parser source code for collaboration?

Hey, Bryan.

I've thought about it, and I'm not immediately opposed to it.  But, it
would take some effort to tidy things up, and to set up a svn
repository, before I could do that.  I'm also not eager to give the
commercial services any freebies.  And lastly, no one has expressed a
real interest in contributing before.

If you're really serious about it, I'll put that on my list of things to do.

Is there anything in particular you'd be interested in doing/improving
with the parsers?

--
- Joshua Tauberer

http://taubz.for.net

"Unfortunately, we're having this discussion. It's too bad,
because guess who listens to the discussion: the enemy."

Joshua,

One big thing I'd like to do is get the parser to output into a SQL
database, as opposed to just XML.

Besides that, I'd like to take a look at the possibility of making
some incremental improvements... for example, perhaps it would be
possible to keep track of who is the chair at any given time.

Another thing I noticed is sometimes narrative actions in the CR
source don't get included in your XML.  I'd like to see about getting
those all in there.

If you're game for it, don't worry about tidying up the source code.
I won't hold any GOTO statements against you. :)  I'd just rather dig
in asap.

Let me know.

-Bryan

On 2/20/06, Joshua Tauberer / GovTrack.us <tauberer@...> wrote:
> Hey, Bryan.
>
> I've thought about it, and I'm not immediately opposed to it.  But, it
> would take some effort to tidy things up, and to set up a svn
> repository, before I could do that.  I'm also not eager to give the
> commercial services any freebies.  And lastly, no one has expressed a
> real interest in contributing before.
>
> If you're really serious about it, I'll put that on my list of things to do.
>
> Is there anything in particular you'd be interested in doing/improving
> with the parsers?
>
> --
> - Joshua Tauberer
>
> http://taubz.for.net
>
> "Unfortunately, we're having this discussion. It's too bad,
> because guess who listens to the discussion: the enemy."
>
>
>
> Yahoo! Groups Links
>
>
>
>
>
>
>


--
http://www.MyCongress.org/ -- coming soon

--- In govtrack@yahoogroups.com, "Bryan Helmkamp" <helmkam1@...> wrote:
> One big thing I'd like to do is get the parser to output into a SQL
> database, as opposed to just XML.

Ok...

> Besides that, I'd like to take a look at the possibility of making
> some incremental improvements... for example, perhaps it would be
> possible to keep track of who is the chair at any given time.

I haven't even noticed that information in the record.

> Another thing I noticed is sometimes narrative actions in the CR
> source don't get included in your XML.  I'd like to see about getting
> those all in there.

Right.  They often go on for pages with the text of legislation,
amendments, and roll calls that quickly clutter up the main purpose of
the files.  I'm sure I could just set a flag to keep them in, although
I wouldn't want to do that for GovTrack.

> If you're game for it, don't worry about tidying up the source code.
> I won't hold any GOTO statements against you. :)  I'd just rather dig
> in asap.

Heh, well, it's a bit more than that.  The person-name-to-id system is
tied to a database which you wouldn't have access to, for instance.
The roll call votes stuff (which you may not be interested in now) is
tied to various data files and programs to generate the maps.

(For some reason I didn't get your message, although Yahoo says it
sent it, so I'm replying via Yahoo.  Strange.)

- Josh