Nations Political Pulse, Taken Using Net Chatter
By JOSHUA BRUSTEIN NY Times
Published: October 31, 2010
When a Rasmussen poll last month showed
Representative Roy Blunt opening a double-digit
lead over Robin Carnahan in their Senate campaign
in Missouri, John Hancock was not surprised.
Mr. Hancock, a political consultant advising the
Blunt campaign, had seen a similar shift in
public opinion days earlier, through a software
tool that analyzed the language being used in
conversations about the campaign on social
networking sites, blogs and other online conversations.
He said this technique, known as sentiment
analysis, would soon be a part of every campaign
he works on, because it helps him determine
quickly which messages are resonating with
potential voters. You get a real sense of whos
carrying the day, he said. It affects the advice youre able to give.
Online organizing techniques have been rapidly
adopted by the political world, and they played
an important role in President Obamas victory in
2008. Now, campaigns and the news media are
becoming convinced that the Internet can also be
mined systematically for useful data about public
opinion. The New York Times has a tool that
monitors Twitter for posts about candidates.
Businesses were quick to embrace sentiment
analysis to monitor the performance of their
brands. The programs scrape online messages for
mention of certain products, then determine
whether the language is positive or negative.
While political hands say there is interest in
adapting these techniques to campaigns, the pace
is more cautious. The analytical rigor can vary
widely, they say, and there is uncertainty about
how to use these tools. But Mr. Hancock and
others say that such techniques could become common by 2012.
Technology companies that do sentiment analysis
say their tools are already providing useful
results. Linguamatics, a British company,
analyzed posts from more than 130,000 Twitter
accounts to gauge public opinion during the
British elections this spring. The companys
analysis yielded similar results to traditional
political polling, and predicted within one point
the percentage of votes the Conservative Party would win.
Crimson Hexagon, a technology company in
Massachusetts, analyzed expressions of public
sentiment across the country about the oil spill
in the Gulf of Mexico. Its analysis showed that
people who lived near the gulf had a lower
tendency to assign blame, focusing instead on the
logistics of the relief efforts.
The ability to provide data on public opinion in
real time is a primary attraction of sentiment
analysis. Because the technique passively
monitors conversations, it can track which ideas
develop organically, something that is unlikely
to happen in traditional polling when respondents reply to specific questions.
Changes in the way people communicate
particularly the increasing number of people who
do not have landline telephones also present
challenges for traditional polling. The Pew
Research Center released a report in October
finding that polls that do not use cellphones in
their samples can overestimate support for
Republican candidates by four to six percentage
points. Some advocates for sentiment analysis see
this as evidence of a need for new techniques.
Were not necessarily seeking to replace
immediately, in 2012 the traditional mechanism.
But its got to have a seat at the table, said
Michael Urban, who worked on several Republican
campaigns and on polling for Mr. Hancocks
political consultancy, before starting
Globalpoint, a start-up that develops sentiment
analysis tools for use in politics.
To be sure, there can be pitfalls in divining
public opinion from online musings. The people
who post their political views online are not a
representative sample of the population, either
demographically or in their level of political
engagement. Further, the messages often come with
little or no information about the person who posted them.
Sentiment analysis tools also have a tin ear for
sarcasm, and are easily distracted by active but
irrelevant conversation. During the recent
British elections, Linguamatics recognized a huge
spike in positive language about David Cameron,
the Conservative Party candidate for prime
minister, in the first moments of a televised
debate. It was almost entirely because of a
message on Twitter that had been resent many
times that jokingly claimed that a television
network had already declared Mr. Cameron the winner of the debate.
But the lack of scientific precision does not
mean that this technique is not useful, say those
familiar with it. Sentiment analysis just has to find its role.
On election night on Tuesday, CNN plans to use
sentiment analysis to identify and track themes
that gain traction online. These could be
opinions on the political parties general
support for the Tea Party movement, say or on the issues.
But David Bohrman, CNNs Washington bureau chief,
says that the most attractive part about the
system has been its ability to monitor the
conversations that are taking place
independently, rather than having to frame the
issues by asking direct questions.
Were waiting to see what theyre going to say, he said.