World's publishers face off against Google: It's getting ugly

I had a feeling this would be the end result. At first there was co-operation and pleasantries exchanged between the media publishers and Google, and then it all went sour. Online publishers and newspapers appear to be heading for a face-off with search engine behemoth, Google.

On Thursday, online publishers and print media in the guise of the powerful World Association of Newspapers (WAN) issued a rather terse statement, calling on Google “to respect the rights of content creators” and embrace a new access protocol for search engines visiting websites, known as ACAP.

In the statement, WAN president Gavin O’Reilly, implied that Google’s reluctance to accept ACAP is as a result of “its own commercial self-interest”, adding that the search engine behemoth should “not glibly throw mistruths about”. This is the first salvo in what will probably become a key battle between Google and media players in the next few years.

ACAP (Automated Content Access Protocol) is a proposed search engine protocol for accessing publisher websites, created by the publishing industry under WAN’s leadership. Publishers world-wide have started implementing ACAP on their websites. ­WAN says that publishers in 16 countries are known to have already implemented it. The consortium includes news agencies, book and magazine publishers, libraries and search engines as well as newspaper publishers.

WAN unhappy with a statement by Google European executive
According to the WAN statement, Google European executive Rob Jonas (who was referred to as “Ron” instead of “Rob” in WAN’s official PR) implied that Google would not embrace ACAP. He was quoted as saying this week that the current robots.txt protocol “provides everything most publishers need”, implying that the search engine was happy with the status quo.

Rob Jonas was at the WAN conference in Cape Town last year and spoke at the Digital Round Table, which I chaired. He’s a nice, approachable guy, reminding me a bit of James Bond, actually. If I remember correctly, his presentation was well received, although Google was both slammed and praised at the conference.

It’s interesting that WAN are firing off at Jonas over his recent comments. It was probably just some off-the-cuff comment he made sometime, somewhere, presumably at some conference. So it might appear at first glance to be an overreaction from WAN, but more likely this is symptomatic of a gradual cooling in relations between the two parties that has reached this point.

How ACAP works
The idea behind ACAP is that publishers would place code on their servers that would control search engine access. Currently the robots.txt method does this, but WAN is saying that it is too simplistic and does not give publishers enough options. Furthermore, it’s not a gatekeeping mechanism that online publishers have a stake in — the current access protocol was something imposed on them by search engines years ago. Here is an example of the ACAP protocol implemented in the robots.txt of the UK’s Times Online (You will see the standard allows for more parameters).

Stormy relationship
So now it appears that relations, which started off rather cordially, have broken down. Either the parties will enter into some serious arbitration or its going to go legal. Fundamentally I think this is all about money. (What else would it be about.) WAN will tell us now that it is about controlling access and respecting their rights. But essentially controlling access will mean that publishers will eventually be in a position to charge Google to crawl or list their content, even in aggregated form.

The relationship between WAN and Google has always been stormy. WAN President Gavin O’Reilly a few years back slammed what he called the “Napsterisation” of content — pointing to the fight between the music download service and the record industry. O’Reilly went on an all-out offensive, saying in interviews that we’re dealing with “basic theft” and “kleptomania” here. Rather strong stuff. I hope it’s not WAN’s intention to tacitly compare itself to the notorious RIAA.

Since these fighting words and the arrival of ACAP, relations with Google appeared to become cordial again. The adults decided to sit around a table and there were sweet words of co-operation for a while. Now that ACAP is becoming a reality, it now looks like the temperature has risen. In many respects Google has become a victim of its own success, and now the rest of internet wants an even bigger piece of the pie. The search engine giant is getting bigger and scarier everyday, or as Wired magazine puts it, going from “guerrilla startup to 800-pound gorilla”.

Two sides of the story
You can see both sides of the story here. The argument against: Why should Google aggregate and list content it does not pay for? This is content that publishers originate and there is a cost associated with it. The argument for: On the other side — how else is Google supposed to behave if it is to be a search engine? It has to aggregate headlines and blurbs, in order to send traffic through to websites. Arguably Google News is a competing news brand, using content that belongs to other news sites. But the search engine has also been very careful NOT to monetise Google News by listing Adsense on it — a move which would infuriate publishers who would claim that Google is directly profiting off content that is not theirs.

WAN also needs to be careful here. Although it represents a powerful publishing lobby of newspapers and online publishers, the publishing community is anything but united on this issue. Google may aggregate publisher content, but it is also a huge source of traffic and in some cases, revenue, for many publishers. Many online publishers would be reluctant to give that up, especially smaller and mid-size publishers that rely on Google heavily.

Also see: Why we love and hate Google and Google slammed, Google praised at WAN conference.

Comments (4)

  1. The following is extracted from a media update from the European Publishers Council I received yesterday (I’ve forwarded you the complete email, in case you’re not on that mailing list):


    4.1 Search engines outside EU must stick by EU rules

    European data privacy regulators have said that Internet search engines based outside Europe must also comply with EU rules on how a person’s Internet address or search history is stored. EU rules state that someone must consent to their data being collected and individuals must be given the right to object or verify their information.

    Any ISP with “an establishment” in any EU Member State or indeed any ISP that uses automated equipment from an EU Member State, must comply.

    Meanwhile, European privacy regulators are set to impose tighter restrictions on the way search engines such as those of Yahoo and Microsoft keep customer data. Peter Schaar, Germany’s federal data protection commissioner and chairman of the Article 29 working party that advises the European Union on privacy policy, has accused search engines of keeping data too long. Mr Schaar has said that the working party also plans to investigate targeted advertising in terms of data protection. This could be another potential blow to companies building businesses around their ability to profile and deliver tailored advertising to consumers.”

    Thursday, March 13, 2008 at 6:02 pm #
  2. Andy Hadfield wrote::

    Don’t know if you saw that conspiracy vid from a while back. Googlezon. Tad freaky how it’s starting to come true… The Great Media Wars of 2010. Followed obviously (cue dramatic drumroll) by newspaper shutting down and the Washington Post becoming a PRINT ONLY publication for “refined taste”.


    Friday, March 14, 2008 at 11:16 am #
  3. Nick wrote::

    I think the Newspapers ought to have the odds in their favour. They are the originators of the content after all. And since when did we live in a world where we could have a free lunch. That said, everyone uses the internet and the internet is free…isn’t it?

    Friday, March 14, 2008 at 3:41 pm #
  4. Mark Bide wrote::

    Thank you very much for your thoughtful commentary on recent developments relating to ACAP. I hope that you will allow me, as Project Director of ACAP, the opportunity to respond to one or two of the points which your raise?

    First of all, the issue of the robots exclusion protocol (REP). Why are publishers dissatisfied with it as a mechanism for managing policies relating to access and reuse of their content? You mention one of the problems – the lack of any proper governance mechanism. It is an orphan which never got further than being a draft specification; the specification is no longer managed, has various proprietary extensions and is very inconsistently implemented. Unlike the major search engines, the majority of those who crawl the web for content simply ignore REP completely, helping themselves to publishers’ content without respect even for the very simplistic policies which publishers can express in REP.

    However, there are other issues which are equally troubling.

    REP was not devised to manage policies relating to the content of the web but rather to manage problems relating to bandwidth (a rather greater challenge in 1994 than it is today). The fact that this is the only protocol currently available for managing access policies is a clear indication of the problem – both online search and online publishing have changed out of all recognition since 1994. The time has undoubtedly become ripe for an extensive overhaul – which is unlikely to be led from anywhere other than the content industries.

    ACAP would certainly have preferred not to have had to propose extensions to REP, but we have done so because we were advised by the search engine community that this was the only possible way forward from their point of view.

    What we are attempting to achieve is straightforward enough: to offer content owners more choice in terms of the policies which they set for access to and reuse of the content that they own. This is an issue which is directly echoed in an interview with Tim Berners Lee reported this morning on the BBC. He is talking about personal information, but the point he makes is exactly the same as the one that publishers feel about their content: “It’s mine – you can’t have it. If you want to use it for something, then you have to negotiate with me. I have to agree, I have to understand what I’m getting in return.”

    Publishers fully recognise that “negotiation” – in terms of one-to-one human intervention – is not a mechanism which is appropriate for internet scale commerce in content. ACAP is developing an approach built on machine-readable and machine-interpretable policy which is appropriate to the complex commercial environment which the web now represents.

    You say that this struggle is “fundamentally about money” – something ACAP has never for one moment questioned. ACAP is not in any way embarrassed about having commercial drivers. This is what we have long said in our FAQs:

    “.Publishers are not ashamed about making money out of publishing – that is their business. They make substantial investments in the creation and distribution of content, and believe that they should be able to make a fair return on those investments. Business models are changing, and publishers need a protocol to express permissions of access and use that is flexible and extensible as new business models arise.”

    Finally, you make the point that many online publishers would be reluctant to give up the traffic that is driven to them by Google and the other major search engines. Neither ACAP nor WAN has an agenda that suggests that anyone should give up any business relationship which is beneficial to them. ACAP is an enabling technology, intended to give publishers choice about the way in which their content is used – it does not prescribe the business models that publishers should adopt, but rather seeks to make a multiplicity of business models possible that are currently not realistic.

    Mark Bide 18 March 2008

    Tuesday, March 18, 2008 at 11:28 am #

Trackback/Pingback (1)

  1. […] It was under a week ago, as reported here, that WAN released a terse statement calling on Google “to respect the rights of content creators and embrace ACAP”. It criticised Google for acting in “its own commercial self-interest” by its apparent reluctance to accept the new search engine protocol known as ACAP. Gavin O’Reilly, President of WAN and Chairman of ACAP, did not mince his words, warning the search engine behemoth not to “glibly throw mistruths about”. […]