Google has made its first concession to those who accuse it of exploiting the content of news providers by allowing media companies to restrict internet users’ free access to paid-for material.

The U.S. company’s decision to restrict users’ ability to bypass paywalls through its news aggregation service Google News comes as Rupert Murdoch prepares to introduce charging at his papers globally.

Mr. Murdoch, the chairman and chief executive of News Corporation, has been turning up the heat on Google in recent months and this week used an appearance before U.S. regulators to repeat his charge that news aggregators who pay nothing to the producers of content are guilty of “theft”.

But Google’s senior business product manager, Josh Cohen, said publishers erecting a pay wall needed Google’s search engine to reach an online audience more than ever.

“Each new click, each visit, each page view, each reader they get, represents a business opportunity,” he said in an interview with the Guardian. “I would argue that if you are putting up a paywall, getting traffic and being discovered is even more important because you have got a smaller set of users who are potentially willing to pay. Discovery is just as important.”

Google’s change of stance relates to Google News, which allows users to search for specific news stories and groups relevant versions from different papers together in a list. Until now users have been able gain access through Google News to individual stories that would otherwise be locked behind a paywall under its First Click Free programme. By repeatedly searching on the aggregation service they can therefore avoid paying for access to newspapers that require online subscriptions, such as the Financial Times and Mr. Murdoch’s Wall Street Journal.

Limiting access

From now on, they will be able to gain free access to five stories a day before coming up against a requirement to register or subscribe to websites with paywalls.

Announcing the move on a company blog, Mr. Cohen said publishers would still be able to charge for their content and make it available via Google. “The two aren’t mutually exclusive,” he said.

Google would treat as “free” any preview pages — usually the headline and first few paragraphs of a story — from subscription websites, and then label the stories as “subscription” on Google News, he said.

“The ranking of these articles will be subject to the same criteria as all sites in Google, whether paid or free,” he added. “Paid content may not do as well as free options, but that is not a decision we make based on whether or not it’s free. It’s simply based on the popularity of the content with users and other sites that link to it.”

Industry insiders said the changes represented the first time Google had acknowledged that it had been potentially causing harm to publishers but felt they failed to address fundamental concerns about some of the search company’s activities.

Mr. Murdoch and his lieutenants believe that Google has exploited the presence of externally produced quality content on its search index to generate huge amounts in advertising revenue for itself. Rather than submit to Google’s power to reach readers in an attempt to build a huge free online audience, they think newspapers should instead concentrate on making money from the loyal minority of readers who can be persuaded to subscribe online.

Few in the industry think that Mr. Murdoch will want his newspapers to come off Google altogether, as he recently threatened, given its ability to provide a shop window even for paid-for content. His rhetoric is seen by some as part of a strategy aimed at winning regulatory assistance to curb Google and possibly to pave the way for an advantageous deal with it.

Still, many in the industry are sceptical that Mr. Murdoch’s paywall plan will work for general news, now that free access to online newspapers has become widely established. Nevertheless, many traditional news organisations, even those that have for now decided to remain free online are unhappy about the way Google News works and would like to see it offer some payment for including their content. — © Guardian Newspapers Limited, 2009

More In: Comment | Opinion