Edward Summers wrote:
   > "The Government Printing Office is setting an aggressive timetable
> for its massive project to digitize nearly every federal document
> published since the birth of the nation. "

Very neat.

--
- Joshua Tauberer

http://taubz.for.net

** Nothing Unreal Exists **

Hello,

Congrats on all the work this group is doing. GovTrack has impressed
me quite a bit. At Joshua's suggestion, I'm posting here to see
if anyone has interest in working with my team on a related project
we're beginning.

My team makes campaign software for activist organizations and
progressive candidates, called Campaign Engine. We're proud of the
comprehensiveness of our product, which is also very easy to use. It
is LAMP-based, but it is a commercial product. You can read about it
here: http://www.mediamezcla.com/products/campaign/

For our next version, we want to implement a "find my legislator"
feature -- ideally that works on local, state, and federal levels for
50 states.

We're looking for coders who can use TigerLine census data to create
matching tables between ZIP+4 and local, state, and federal
legislative districts. (Or, if you recommend a different approach,
we'd consider it.) This is a for-pay, contract assignment.

Anyone who's interested should email me at bschaffer@....
Please include your resume, rates, and other relevant work information.

Thanks again for the work you're doing.

Best,
Ben Schaffer
Media Mezcla LLC

Hey, guys.

Today I had an article published on xml.com about bringing government
data onto the semantic web.  It should be the first in a series of
similar articles.

http://www.xml.com/pub/a/2006/02/08/govtrack-us-public-data-semantic-web.html

Hope not everyone has lost interest in opening up access to the government!

--
- Joshua Tauberer

http://taubz.for.net

** Nothing Unreal Exists **

--- "Joshua Tauberer / GovTrack.us"
<tauberer@...> wrote:
Not at all.
--Peggy

>
> Hope not everyone has lost interest in opening up
> access to the government!
>
> --
> - Joshua Tauberer
>
> http://taubz.for.net
>
> ** Nothing Unreal Exists **
>
>
>

On Feb 9, 2006, at 9:13 PM, Joshua Tauberer / GovTrack.us wrote:
> http://www.xml.com/pub/a/2006/02/08/govtrack-us-public-data-
> semantic-web.html

Nice work Joshua!

> Hope not everyone has lost interest in opening up access to the
> government!

The interest is still there, but my personal motivation for screen
scraping data from local Illinois data has waned due to other
projects. Thanks for the kick in the pants though. You are doing
important work.

//Ed

Hello,

I was looking at the raw XML data available on the GovTrack.us site,
and I couldn't find XML versions of the debates from the Congressional
Record.

Are these available?  Where would they be located?

I've been trying to build a parser for that data for some time, and it
has proved to be rather difficult.

Thanks,

-Bryan

alphasigbryan wrote:
> I was looking at the raw XML data available on the GovTrack.us site,
> and I couldn't find XML versions of the debates from the
> Congressional Record. Are these available?  Where would they be
> located?

Hi, Bryan.  Yep, they're here:

http://www.govtrack.us/data/us/109/cr/

> I've been trying to build a parser for that data for some time, and
> it has proved to be rather difficult.

Yeah, that was for sure the oddest text to parse, and I find bugs with
the parser pretty often.

--
- Joshua Tauberer

http://taubz.for.net

** Nothing Unreal Exists **

Bryan Helmkamp wrote:
> Actually, open doing some more searching, I found what I was looking for
> in the "cr" directories.  It appears that the newest data is a couple
> weeks old though.  How often does GovTrack update?  Are the updates
> found in the /data/ directory immediately?

Uh, yeah, apparently I forgot to comment something out after making some
changes and so GovTrack hasn't been fetching them for a few weeks.  Doh.

They're downloading now (and appear immediately in that directory;
updates are daily, when I don't mess things up).

Thanks for pointing this out!

--
- Joshua Tauberer

http://taubz.for.net

"Unfortunately, we're having this discussion. It's too bad,
because guess who listens to the discussion: the enemy."

Joshua,

Have you ever thought about opening the parser source code for collaboration?

-Bryan

On 2/20/06, Joshua Tauberer / GovTrack.us <tauberer@...> wrote:
> Bryan Helmkamp wrote:
> > Actually, open doing some more searching, I found what I was looking for
> > in the "cr" directories.  It appears that the newest data is a couple
> > weeks old though.  How often does GovTrack update?  Are the updates
> > found in the /data/ directory immediately?
>
> Uh, yeah, apparently I forgot to comment something out after making some
> changes and so GovTrack hasn't been fetching them for a few weeks.  Doh.
>
> They're downloading now (and appear immediately in that directory;
> updates are daily, when I don't mess things up).
>
> Thanks for pointing this out!

Bryan Helmkamp wrote:
> Have you ever thought about opening the parser source code for collaboration?

Hey, Bryan.

I've thought about it, and I'm not immediately opposed to it.  But, it
would take some effort to tidy things up, and to set up a svn
repository, before I could do that.  I'm also not eager to give the
commercial services any freebies.  And lastly, no one has expressed a
real interest in contributing before.

If you're really serious about it, I'll put that on my list of things to do.

Is there anything in particular you'd be interested in doing/improving
with the parsers?

--
- Joshua Tauberer

http://taubz.for.net

"Unfortunately, we're having this discussion. It's too bad,
because guess who listens to the discussion: the enemy."

Joshua,

One big thing I'd like to do is get the parser to output into a SQL
database, as opposed to just XML.

Besides that, I'd like to take a look at the possibility of making
some incremental improvements... for example, perhaps it would be
possible to keep track of who is the chair at any given time.

Another thing I noticed is sometimes narrative actions in the CR
source don't get included in your XML.  I'd like to see about getting
those all in there.

If you're game for it, don't worry about tidying up the source code.
I won't hold any GOTO statements against you. :)  I'd just rather dig
in asap.

Let me know.

-Bryan

On 2/20/06, Joshua Tauberer / GovTrack.us <tauberer@...> wrote:
> Hey, Bryan.
>
> I've thought about it, and I'm not immediately opposed to it.  But, it
> would take some effort to tidy things up, and to set up a svn
> repository, before I could do that.  I'm also not eager to give the
> commercial services any freebies.  And lastly, no one has expressed a
> real interest in contributing before.
>
> If you're really serious about it, I'll put that on my list of things to do.
>
> Is there anything in particular you'd be interested in doing/improving
> with the parsers?
>
> --
> - Joshua Tauberer
>
> http://taubz.for.net
>
> "Unfortunately, we're having this discussion. It's too bad,
> because guess who listens to the discussion: the enemy."
>
>
>
> Yahoo! Groups Links
>
>
>
>
>
>
>


--
http://www.MyCongress.org/ -- coming soon

--- In govtrack@yahoogroups.com, "Bryan Helmkamp" <helmkam1@...> wrote:
> One big thing I'd like to do is get the parser to output into a SQL
> database, as opposed to just XML.

Ok...

> Besides that, I'd like to take a look at the possibility of making
> some incremental improvements... for example, perhaps it would be
> possible to keep track of who is the chair at any given time.

I haven't even noticed that information in the record.

> Another thing I noticed is sometimes narrative actions in the CR
> source don't get included in your XML.  I'd like to see about getting
> those all in there.

Right.  They often go on for pages with the text of legislation,
amendments, and roll calls that quickly clutter up the main purpose of
the files.  I'm sure I could just set a flag to keep them in, although
I wouldn't want to do that for GovTrack.

> If you're game for it, don't worry about tidying up the source code.
> I won't hold any GOTO statements against you. :)  I'd just rather dig
> in asap.

Heh, well, it's a bit more than that.  The person-name-to-id system is
tied to a database which you wouldn't have access to, for instance.
The roll call votes stuff (which you may not be interested in now) is
tied to various data files and programs to generate the maps.

(For some reason I didn't get your message, although Yahoo says it
sent it, so I'm replying via Yahoo.  Strange.)

- Josh

Hi, Josh.

On 2/21/06, Joshua Tauberer <tauberer@...> wrote:
> Right.  They often go on for pages with the text of legislation,
> amendments, and roll calls that quickly clutter up the main purpose of
> the files.  I'm sure I could just set a flag to keep them in, although
> I wouldn't want to do that for GovTrack.

In those cases, what I'd like to do is display it like GovTrack does,
but add a link to view the hidden content if you wish.

> Heh, well, it's a bit more than that.  The person-name-to-id system is
> tied to a database which you wouldn't have access to, for instance.
> The roll call votes stuff (which you may not be interested in now) is
> tied to various data files and programs to generate the maps.

If I had to do this from scratch, I'd have to write a name-to-id
matching system anyway, and I'm not interested in the roll call votes
just yet.

Basically, if you could just dump the relavent portion of source on
me, I've got a lot of free time together and I could get it going.  I
think that would be ideal for both of us, short term.

What do you think?

-Bryan


--
http://www.MyCongress.org/ -- coming soon

Bryan Helmkamp wrote:
> Basically, if you could just dump the relavent portion of source on
> me, I've got a lot of free time together and I could get it going.  I
> think that would be ideal for both of us, short term.

But maybe a very short term.  The next time someone wants to use the
sources, I don't want to go through the pain of re-explaining how it
comes together, how to set up the people db, etc.  Plus there's no way
for us to keep our versions in sync as changes are made (including
changes to the database of people, for instance).

I guess the thing is that opening up the scripts is a low priority for
me (sorry), especially if it's just a short-term solution.  I would much
rather enhance my scripts so that all of the original information makes
it into files downloadable in the data directory, and then you can just
use that.

--
- Joshua Tauberer

http://taubz.for.net

"Unfortunately, we're having this discussion. It's too bad,
because guess who listens to the discussion: the enemy."

GovTrack has been nominated for a Webby Award (http://webbyawards.com/)
in the Politics category.  :)

There's also a People's Voice award, so I hope everyone on the list
votes for GovTrack!

http://webbyawards.com/

--
- Joshua Tauberer

http://taubz.for.net

"Unfortunately, we're having this discussion. It's too bad,
because guess who listens to the discussion: the enemy."

I am writing to the govTrack group to introduce the metavid project that
we are working on.
http://metavid.ucsc.edu

In metavid we have been attempting to archive all house and senate floor
audio/video/text record and make that archive freely available for
re-contextulization research and play. All the system components are
open source and freely available.The participant contributed scripts,
overlays, annotations, and tags are are under free-document licenses
similar to wikipeida. And we encourage the reuse of the public domain
archive material. We are just getting started so the systems for user
contributions are not yet fully implemented.

We use a number of methods to extrapolate meta data on the video feeds
for example we are reading the cspan nameplates and I foresee
integration with govTrack.feeds in the near future ;)

I would recommend that you just visit the site to see what its about and
then email me or metavid@... if you have any questions, criticisms,
or ideas :)

--michael


--
Michael Dale <http://danm.ucsc.edu/%7Edale/>
Social Computing Lab <http://hybrid.ucsc.edu/SocialComputingLab/>
Graduate Student Research
DANM <http://danm.ucsc.edu/> at University of California Santa Cruz
<http://ucsc.edu>

Michael Dale wrote:
> I am writing to the govTrack group to introduce the metavid project that
> we are working on.
> http://metavid.ucsc.edu

Hi, Michael.  Thanks for sharing that with us.  It sounds really amazing.

> We use a number of methods to extrapolate meta data on the video feeds
> for example we are reading the cspan nameplates and I foresee
> integration with govTrack.feeds in the near future ;)

That would be great.  I'd love to also link from various places on
GovTrack (bill pages, people pages, and congressional record pages) to
the relevant videos.

Let's talk more about that.

--
- Joshua Tauberer

http://taubz.for.net

"Unfortunately, we're having this discussion. It's too bad,
because guess who listens to the discussion: the enemy."

Hi Everyone,

I would like to personally invite you all to beta test
www.VotersDomain.Com.

We are a non-partisan political website that tracks and records the
actions, movements, thoughts, opinions, legislative priorities, events
attended – all these and more -- of key political figures in U.S.
politics. We have over 5,000 abstracts of newspaper articles and other
"open source" materials on at least 12 key political figures. Sen.
Clinton alone has over 650 specialized "facts only"
politician-specific news abstracts and we only started tracking her 11
months ago!

I invite you to surf our site and would welcome any suggestions or
comments for improvement.

Please use the following username and password to enter
www.votersdomain.com
Username: govtrack
Password: us

I will soon be inviting individuals and organizations to sponsor the
featured profiles and to sponsor the creation of new profiles as well.

I believe in holding our representatives accountable for their
actions.VotersDomain.com provides that service to the public.

I'm curious to know how you went about setting your site up. I think it is
something that we
as citizens should have available in all aspects of government, especially on
the state level.
On a technical level, if I'm looking for help, as a university student, I could
probably scare
some people up, so what if you could tell me what needs must be addressed on
that level it'd
be a big help (Java, html, xml?). Any help is appreciated. Thanks.

Akshai Singh wrote:
> I'm curious to know how you went about setting your site up. I think
it is something that we
> as citizens should have available in all aspects of government,
especially on the state level.
> On a technical level, if I'm looking for help, as a university
student, I could probably scare
> some people up, so what if you could tell me what needs must be
addressed on that level it'd
> be a big help (Java, html, xml?). Any help is appreciated. Thanks.

Hey, Akshai.  Thanks for bringing this to the list.

At the request of someone else, I happened to have just written a little
thing explaining the basics of how to set up something like a GovTrack
clone:

http://www.govtrack.us/localtrack.xpd

As that says, there are basically two components.  The first is
gathering up all of the relevant information, and what kind of
information you can put together will depend on what the state you
choose makes available online.  I've had success doing this part
programmed in Perl -- it's the type of thing Perl was original designed
for -- but any programming language could do it to varying degrees of
ease.  This isn't a particularly difficult task to program, but it's a
little frustrating because the information is rarely in a format that
makes it easy to collect into a normalized database of some sort.

The Perl programs go out and collect the data from the various
government websites that have the information on a regular basis, on
their own.

The second part is letting users access the information via the web.
(I'd be interested in extending GovTrack itself to incorporate
state-level data, too.)

There's a third part which is sort of... not required in any way, but
something I strongly advocate, which is publishing all of the data
that's collected as RDF so that it can be shared on the
(still-not-making-much-progress) semantic web.  But that's another story.

There's been some interest in creating state-level sites in the past on
this list, about a year ago, but that activity has seemed to have
disappeared.  Maybe this summer we'll see some real progress!

Let me know if there's any way I, or anyone else on this list, can help.

--
- Joshua Tauberer

http://taubz.for.net

"Unfortunately, we're having this discussion. It's too bad,
because guess who listens to the discussion: the enemy."

We just went public beta with www.votersdomain.com . Do you guys think
we should go with web 2.0 or web 1.0 newspaper layout? I think web 2.0
might not be as user friendly as web 1.0. New York Times, Washington
Post are still graphically web 1.0 but their funcionality is at 2.0.
Anyway, I'd greatly appreciate any feedback. Thanks :)

On May 24, 2006, at 12:01 PM, be_self_aware wrote:
> We just went public beta with www.votersdomain.com . Do you guys think
> we should go with web 2.0 or web 1.0 newspaper layout? I think web 2.0
> might not be as user friendly as web 1.0. New York Times, Washington
> Post are still graphically web 1.0 but their funcionality is at 2.0.

Could you explain what you mean by "web 2.0 or web 1.0 newspaper
layout"? I'm familiar with the terms "web 1.0" and "web 2.0" but not
what that might mean for your layout.

James.

The layout we're currently using does not follow the
"classic" newspaper-type layout that New York Times,
Washington Post, or the other older well known news
sites use. People are used to reading the news in the
"old fashion" way. I'm trying to figure out if we are
friendly enough for the older generation of news
readers to read.

Here some of my questions for everyone:
1. Is our site easy to navigate?
2. It it easy to figure out that we are actually
keeping a live political dossier on the individuals we
are currently profiling? (THANKS to Josh we are now
trying to figure out how to add legislations in one of
our tabs. However, we are not as knowledgeable as Josh
so this is going to be slow in the making. Kudos to
Josh! :)
3. How do we make our site even more appealing?

Anyway, any suggestions and comments will be
appreciated. Thanks James:)

francis


--- James Stewart <lists@...> wrote:

> On May 24, 2006, at 12:01 PM, be_self_aware wrote:
> > We just went public beta with www.votersdomain.com
> . Do you guys think
> > we should go with web 2.0 or web 1.0 newspaper
> layout? I think web 2.0
> > might not be as user friendly as web 1.0. New York
> Times, Washington
> > Post are still graphically web 1.0 but their
> funcionality is at 2.0.
>
> Could you explain what you mean by "web 2.0 or web
> 1.0 newspaper
> layout"? I'm familiar with the terms "web 1.0" and
> "web 2.0" but not
> what that might mean for your layout.
>
> James.
>


__________________________________________________
Do You Yahoo!?
Tired of spam?  Yahoo! Mail has the best spam protection around
http://mail.yahoo.com

Since scraping information is a common need in the govtrack space,
when I saw this I thought it might be of interest:

  http://www.perl.com/pub/a/2006/06/01/fear-api.html
  by Yung-chung Lin
  June 01, 2006

Thanks for all the wonderful work out there to increase the
transparency of the government, etc.

Neal McBurnett                 http://mcburnett.org/neal/
Signed and/or sealed mail encouraged.  GPG/PGP Keyid: 2C9EBA60

Votersdomain.com looks quite nice, easy to find my way around. The
profiles on the members are also very good. The design makes me want
to stay on the site rather then leave the site, which is the case with
many large-content based web sites. Good job!