Tech algorithms that produce journalism

In the U.S., algorithms are already reporting the news. What might sound like a futuristic setting is already becoming reality.

March 31, 2010 11:31 pm | Updated 11:32 pm IST

Journalistic texts are characterised by a certain structure that algorithms can be programmed to imitate. The first tests still read or hear like early prototypes, but they're already around in sports journalism, with finance or local news to come next.

In the U.S., two different projects have started work on algorithm produced journalism. Last week the sports statistics website StatSheet announced a plan to produce completely automated sports content as of this summer. The algorithm produced content will take the form of blogs, with a target that at least 90 per cent of the readers should think the content was created by a human.

And in a partnership with the Medill school of journalism, the Intelligent Information Laboratory of the McCormick School of Engineering at Northwestern University has developed an algorithm called StatsMonkey that publishes game stories.

Automated journalism can basically be understood as search algorithms programmed to look out for certain key findings. then to put them into a certain structure. For a report on a football game for example, the StatsMonkey calculates the narrative based on the numerical data.

Using the score, the algorithm captures the overall dynamic of the game, highlights the key plays and key players, looks for quotes, and generates a text out of these elements. In addition, it configures an appropriate headline and a photo of the most important player in the game — and there goes a very rough sketch of a sports article.

Thus: Michigan State silences Notre Dame, 3-0 SOUTH BEND, Ind.— Tony Bucciferro put the Michigan State Spartans on his back Sunday and spurred them to a 3-0 win over the Notre Dame Fighting Irish (7-11) at Frank Eck Stadium.

Bucciferro kept the Fighting Irish off the board during his nine innings of work for Michigan State (12-4). He struck out five and allowed one walk and three hits.

Senior Matt Grosso was not able to take advantage of a big opportunity for the Irish in the ninth inning.

After freshman Frank Desico walked, Ryne Intlekofer doubled and Ryan Connolly was hit by a pitch, the Fighting Irish were trailing by three when Grosso came to the plate against Bucciferro with one out and the bases loaded, but he flew out.

Brandon Eckerle was perfect at the plate for the Spartans. He went 4-4 at the dish. Eckerle singled in the first, third, fifth and ninth innings and walked in the seventh inning.

Michigan State scored in two innings to claim the victory. The Spartans scored one run in the first and two runs in the third. In the first, senior Eric Maust gave up one run on a double by Jeff Holm. In the third, Maust gave up one run on a single by Holm. Later that inning, a run came in when Bo Felt reached on a fielding error by third baseman Adam Norton.

Maust took the loss for Notre Dame. He went six innings, gave up one walk, struck out three, and allowed three runs. Michigan State's next game is on Friday, March 26 at Oakland.

As programming semantics got better and better in the recent years, automated journalism will become more widely available.

“Sports is an unbelievable ground for this because it's data intensive,” says Kristian Hammond, co-director of Intelligent Information Laboratory in Illinois. “The system knows how to go off and find information, it knows how to find quotes, it knows how to collect data, but then a traditional journalist has to bring his or her perspective to that story. It will only provide journalists with a starting point.” Both projects emphasise that they are working in areas where journalists aren't working.

The Lab in Illinois for example is testing its StatsMonkey algorithm in a pilot with The Big Ten Network which is dedicated to covering college and university sport. “We are the premier publisher of women softball stories,” says Hammond.

The Intelligence Information Laboratory is also interested in programming algorithms to cover local stories. As the local news outlets are struggling to stay alive, they might have better chances if they can expand their news coverage, to additionally expand their advertising, Hammond says. “We see it as an engine that is increasing the scope what is out there and what is publishable.” Apart from StatsMonkey, which is focused on data-intensive information, the lab also programmed a system that automatically generates a virtual show designed to be funny, focusing on light news like celebrity gossip or movie reviews. The system, supported by the National Science Foundation, collects, parses, edits and organizes news stories and then passes the formatted content to artificial anchors for presentation.

The outcome is sometimes barely comprehensible, but gives a rough idea of what is possible. Picking up opinions using the comments of people, the anchors have a dialogue to balance the pros and cons. If everybody likes the film, they talk about different aspects of it.

The programs are just early prototypes, but will improve quickly with the further development of intelligent semantics. The team of the Intelligence Information Lab is already working on a couple of related projects — Brussell, for example, helps people track developments in ongoing news situations, and Beyond Broadcast is watching television with the user to be able to search for deeper content when asked.

“We know enough intelligent semantics to guide intelligent information systems. We don't want to give them a list of links, so we started working on machine generated content. The next step is finance where we are often looking at data and raw numbers. You can create a graph, or you can write a story out of that,” says Hammond.

While the first prototypes stutter a lot, it is likely that algorithms will change journalistic tasks in the long term, although they won't replace journalists, just as much as spell-checking programmes haven't replaced anybody.

“As far as I can tell, journalists are terrified and needlessly so,” says Hammond.

In the future, writing might not be something anymore that is entirely done by humans, which will surely be debated — and necessarily so.

Apart from the man v machine issue, there are a lot of topics on the table. Should it be made transparent if a text is written by a human or an algorithm? Who controls what the algorithms finds? Is an algorithm more or less open to influence than a journalist? And as the algorithm partly uses what was already written, what happens with copyright? And last but not least, assumed the programming is getting better: do algorithms steal the work of journalists — or help them to cope with information overload? — © Guardian Newspapers Limited, 2010

0 / 0
Sign in to unlock member-only benefits!
  • Access 10 free stories every month
  • Save stories to read later
  • Access to comment on every story
  • Sign-up/manage your newsletter subscriptions with a single click
  • Get notified by email for early access to discounts & offers on our products
Sign in

Comments

Comments have to be in English, and in full sentences. They cannot be abusive or personal. Please abide by our community guidelines for posting your comments.

We have migrated to a new commenting platform. If you are already a registered user of The Hindu and logged in, you may continue to engage with our articles. If you do not have an account please register and login to post comments. Users can access their older comments by logging into their accounts on Vuukle.