Re: [govtrack] Local/State Gov
> If you could head up getting Calif. political dataI'm in the process of gathering California State
> organized, that would
> be fantastic. We should talk more about the details
> of that so you can
> avoid some of the mistakes I've made, but the
> general approach I suggest is:
Assembly and Senate info now. I found a brutally slow
anonymous FTP site to get almost everything I need
(ftp://leginfo.public.ca.gov/pub). Most of the data is
available in HTML format (complete with
<strike></strike> tags for removing content and
<em></em> for adding content to existing statues). I'm
using Perl and HTML::Parser to do most of the dirty
work. I've been able to find text data back to the
93-94 session (although the older data doesn't use
strike and em tags). BTW - using strike and em to
denote changes works great visually maybe that might
work well for govtrack.us' RSS feeds too.
> Get a list of the politicians involved and assignI've thought a little about this lately. I think we
> them all ID numbers.
need to be careful here. If we want to eventually
merge federal and state (and local) data we have to
prevent duplicate IDs for real people. Eureka! We can
just use each politicians SSN! That'd be an excellent
unique ID. haha j/k.
I've seen your people.xml and it looks like your IDs
range from 300000-300159 and 400000-400661. How did
you pick those? How should I pick mine in such a way
that they don't overlap with yours and politicians
from other jurisdictions? Doing this right the first
time will help potential problems (say when we join
databases) in the future.
> a format suitable for sharing federal-level data.SOAP?
Also maybe people and roles should be in seperate
Maybe make people.xml read:
<street>123 Main st</street>
<street>321 Main Ave</street>
Then have a roles.xml:
<!-- Judicial and Executive someday? -->
<ended why="RIP Matsui">2005-01-02</ended>
One person entry for every person. One role entry for
every position in government (along with the list of
people who have held that position).
Do you Yahoo!?
Yahoo! Mail - You care about security. So do we.
- Scott Beardsley wrote:
> I'm using Perl and HTML::Parser to do most of the dirtyOkay, good.
> BTW - using strike and em toI'm not tracking changes at that level of detail now. It's a little bit
> denote changes works great visually maybe that might
> work well for govtrack.us' RSS feeds too.
beyond the scope of what I think people would generally find useful.
> If we want to eventuallyWe're probably going to have to go through a few attempts at assigning
> merge federal and state (and local) data we have to
> prevent duplicate IDs for real people.
common IDs before we get a good system, so I wouldn't worry about it for
now. We can each map our own ID systems to a common naming system later.
> I've seen your people.xml and it looks like your IDsI actually just picked up the IDs that www.opengov.us (now defunct) was
> range from 300000-300159 and 400000-400661. How did
> you pick those?
using, two summers ago. There's no rhyme or reason to the ID assignment
> > a format suitable for sharing federal-level data.RDF would be more appropriate, and this is what I'm looking into now.
See http://w3.org/TR/rdf-primer/ (I haven't read the whole thing myself.)
Because I want to go with RDF, it would be most natural to identify
people with URI's, e.g. I could be:
And Rep. Mike Rogers of Alabama could be:
The actual URI itself doesn't matter, so long as we can agree on a
system of creating them. A URI is slightly more cumbersome than a
numeric ID, but it's more transparent. You have a good idea who a URI
refers to just by looking at it.
Something else to keep in mind is that we don't necessarily need to
agree on a single URI for each person. If we find out we've assigned
two URIs to the same person, we can annotate one URI with the reference
to the other with something like a "this person is the same as this
There's more to be said about this, but I'll come back to it in the future.
> Also maybe people and roles should be in seperate files?Not too important, as long as the information is in there somewhere.
- Joshua Tauberer
** Nothing Unreal Exists **
- --- In firstname.lastname@example.org, Aaron Huslage <huslage@g...> wrote:
> I've been working on getting the data for Oregon, however, abstractsTry to email the governor... He seems to be pro open source maybe
> are the only things available online and there are no typed
> transcripts at all.
he'll also be pro open government.
Did you see this yet:
- I'm glad I found this site and am intrigued by the intent of the
originators. Congratulations to this fine organization and its'
I hope that your organization can make good use of the current
opportunity to protect and enhance the citizen. I'm a political
consultant. My business and my clients run campaigns which are all
about citizen control of the government rather than the one other
and reverse option.
I work extensively with government data, and specifically the data
concerning registration of voters---I use voter files on behalf of
my clients and for our various intrusions into the "processes" of
government", which is another way to say "getting votes".
For the last six months, SOLID, I've been lost in a major overhaul
of my business, and my ongoing confusion and despair is all about
the inability to find and pay for the expertise and advice anyone
needs when confronting the myriad technologies now available and
indispensable for those in this field. Frankly, like it would be for
any business, my problem is learning enough myself so that I can
make wise choices in what technical support I need.
Your group can go one of three ways: your proposal, which looks to
be headed in a generally good direction; an indifferent course,
where you either are ineffective because you never do anything or
ineffective because you continually are in the dark about what is
going on, and always are outmaneuvered by more knowledgeable
operatives who for whatever reasons, are seeking something different
than what you think is important; or you could go in the direction
of profiting from what I think is an enormous amount of power and
influence by closing off access to any real information and making
damn sure that a whole lot of intentionally misleading information
is substituted for and passed off as the real thing...and this last
route is well-traveled, as it's been the choice of many, dating to
the country's first day.
I read the prior posts to this list, which included one from someone
in Sacramento (my old home town and a town in which I've done much
campaign work) and another post from someone in Oregon (where I now
live and in which I do extensive campaign work) and I've included
here some fairly lengthy comments, with examples on where to go or
what to do in Sacramento or Oregon, but which apply elsewhere. I've
managed campaigns all over the country, and for more than 25 years
now, and will gladly help anyone with any information they need and
which I might know something about.
If your mission or goal is openness and access by the citizen to
this thing called "government", I'm on your side. And before you
consider that a good thing, ask yourselves "Just exactly who is our
new ally, and what is he trying to get?"
And remember to judge all you do and all that is done or proposed by
others with that kind of general and reasonable examination.
Please first consider the history and intent of all those with whom
you deal and upon whom you rely for guidance or cooperation...for
example: the posts concerning the Governor of Oregon, and the "open
source" generalities, didn't mention any of the more obvious
concerns we all must have about compilation and disposition of
data...and as for Oregon's Governor and government, their history
and current practices concerning such data are chock full of major
problems. That's not to suggest that the Governor of Oregon and thw
whole of that state's government are worse than elsewhere---no, I
will say, however, that not a single state in this nation is
anywhere near good or decent--- and I will also insist that not one
of the existing state-by-state comparisons for "openness" are all
that accurate as yet. And there are many groups which have proposed
and are seeking the kind of openness which your group proposes to
protect and enhance.
Open access to all data which government compiles and/or manages is
a hot debate being openly conducted (though I would put many
qualifications on how to define "open", and herein, I'm not using
Today, there is a real need for people to become engaged in this
debate. And it's gotta be done right now---and the good guys better
have some geeks with them, to translate and to inform--- so that the
non-technically proficient among us don't get into trouble, BIG TIME
and suddenly find that we gave away all kinds of data which is NO
ONE's business just because we thought we were doing the right thing
or because we didn't pay any attention when it was being opened up
to "public" access(at the end I've attached a few lines about some
problems which have erupted during this last election cycle).
I suggest that you start with a review and analysis of all those
individuals and groups who for so long have been doing or attempting
to do what you now propose (since those groups which have goals
similar to your own and with which I am familiar make for a VERY
long list, I've noted only a few here).
And let's all of us also identify the people and groups who are or
likely will be MISUSING that same opportunity.
As for groups you can begin with, one of your obvious tools, as well
as a starting point for any group establishing its' intent or model,
is the Freedom of Information Act. FOIA to put it simply, IS
government information. You will want to consider both the original
intent of FOIA, all of its' revisions over the years, and it's
Especially for those of you in the California, California Voter
Foundation is a place you might go to get an overview of some of the
info now available and to see how various interested parties are
attempting to influence the collection of data and its' ultimate
disposition---it's a private foundation, with an agenda (and EVERY
ONE has an agenda, so learn the agenda of this group, too, and keep
it in mind when evaluating what they propose---when you look into
those who oppose some of what this Foundation advocates, you'll find
many new avenues of inquiry).
Right there in Sacramento you have one of the absolute best
companies of all those which specialize in selling voter data and
enhancements. And learn about all the major players in selling this
data and using it for campaigns.
If I can answer any questions or add anything of use to someone, I
will be glad to help. Let's keep it open and only keep what we have
a right to possess.
- directaction wrote:
> I'm glad I found this site and am intrigued by the intent of theHello, and thanks!
> originators. Congratulations to this fine organization and its'
> recent award.
I'm not sure how much intrigue there could be about GovTrack, unless you
think I might have some ulterior profit motives. My intent was to
create what you see now on the site. That's basically it.
> I hope that your organization can make good use of the currentThat's the general idea.
> opportunity to protect and enhance the citizen.
> Your group can go one of three ways: your proposal, which looks to beNot sure what proposal you're refering to. If you mean the 'long-term
> headed in a generally good direction;
mission' box on the main page of the site, it's a good direction, yeah...
> or you could go in the direction of profiting from what I think is anWell, that's not my intention. That will become clearer in the next few
> enormous amount of power and influence by closing off access to any
> real information
months as I work on open standards for sharing information.
> I've managed campaigns all over the country, and for more than 25 yearsI'm sure people will appreciate that.
> now, and will gladly help anyone with any information they need and
> which I might know something about.
> Open access to all data which government compiles and/or manages is aThat type of information is far beyond the scope of this mail list. All
> hot debate being openly conducted ...
> and suddenly find that we gave away all kinds of data which is NO
> ONE's business just because we thought we were doing the right thing
we're concerned about here is legislative records that are already a
matter of public record and, for the most part, already accessible on
> I suggest that you start with a review and analysis of all thoseWell, as if I have time to do a careful review and analysis of anything. :)
> individuals and groups who for so long have been doing or attempting
> to do what you now propose
> And let's all of us also identify the people and groups who are orI can't disagree more. I have absolutely no concerns about whether
> likely will be MISUSING that same opportunity.
people might misuse the data I publish. Given the type of information
that I'm dealing with, there's simply no harm, without deliberate
misuse, in publishing the truth. And, the same for the other types of
information we've been talking about on this list.
> If I can answer any questions or add anything of use to someone, ILast time I checked, we've got a right to all of the information that
> will be glad to help. Let's keep it open and only keep what we have
> a right to possess.
we've ever talked about here. One might say more than a right to
possess it, a duty to publish it.
- Joshua Tauberer
** Nothing Unreal Exists **