An Open Protocol for Conversations

Google's recent release of Sidewiki, a separate commenting layer provided by the Google toolbar for any webpage, met with less than total support amongst the internet community, especially amongst bloggers and site owners who already host comments and conversation on their site. Jeff Jarvis, author of What Would Google Do?, sees danger:

...Google is trying to take interactivity away from the source and centralize it. This isn’t like Disqus, which enables me to add comment functionality on my blog. It takes comments away from my blog and puts them on Google. That sets up Google in channel conflict vs me. It robs my site of much of its value (if the real conversation about WWGD? had occurred on Google instead of at Buzzmachine, how does that help me?). On a practical level, only people who use the Google Toolbar will see the comments left using it and so it bifurcates the conversation and puts some of it behind a hedge. Ethically, this is like other services that tried to frame a source’s content or that tried to add advertising to a site via a browser...

In a later post Jeff qualified his remarks around what Google should do in this area. As the first comment I made the following remark:

I believe Google should be leading the charge in creating a open *protocol* for conversations. A protocol that can be usefully federated across service providers and lightweight enough so that small-players like standalone blogging software can play their part.

Because the protocol would be open service providers can easily allow users to decorate other conversations with spin-off threads. Content sites could collect and curate part of the conversation that touched their site...

To which Bob Wyman asked:

Can you describe what services you would expect such a protocol to provide or otherwise provide “requirements” for such a protocol?

Here is my reply about what would be required for an open protocol for conversations (or perhaps just the start of a reply). I have not thought about it in detail, so my speculation is somewhat abstract and liable to have problems.

Firstly, let me jump to the end and identify what I think conversations protocol and service should allow us to do:

• Identify a given element (or potential element) of a conversation
  
• Identify new elements of conversation
  
• Find and examine the full-text or full version of the contribution
  
• Understand the direct antecedent, subsequent and possibly sibling contributions. Be able to reconstruct all contributions that led to a specific element, back to the original contribution. Effectively we should able to crawl the conversation in the same way that search engines spider web pages.
  
• Facilitate a distribution and/or federated tracking of elements that maintains the metadata required to identify the antecedent of each contribution.
  
• Allow cross domain and cross media contributions
  
• Allow a new element of conversation to register what antecedents it has.
  
• Reliably identify the author of each contribution in a non-ambiguous fashion.
  
• Allow contribution domains to control contributions on their own domain, but not others.
  

For context on these services (and what they mean), consider my chain of thought (which is, frankly a ramble): We should start by considering what we currently understand a conversation currently is. To me, a conversation is informal collection of contributions from separate authors. These contributions take the form of blog posts, tweets, videos, podcasts, site updates or nearly anything else. To be a part of a conversation a contribution must be framed in reply to one or more other contributions (or for the root contribution, have contributions made about it). Conversations evolve and separate conversations diverge or converge. Conversations begin on one site, or many sites and might be continued on any other site. People join and leave conversations at different times. Conversations could start from any post, tweet, image or video on the web.

What would we like conversations protocol to achieve? We want to track the conversation and communicate its progress. We do not want to be limited about what our conversation is about. Or how we contribute to it. In its basic form, we just want to know what we are replying to, and what is replying to us. We want to be able to be able to understand the context of a contribution by looking at what inspired it and what it has inspired. We recognise that those that host a conversation have a right to curate and ensure a high quality of conversation, but we do not want those hosts to have the final say about what gets said in a conversation, or where it goes.

Given the organic, evolving and changing notion of any conversation, I would not attempt to describe or interact with the entire conversation itself. I think open protocols for conversations would rather operate on one or more contributions. So lets talk about contributions.

Each contribution needs to be identifiable. The easiest way to identify a contribution would be via a URI. For a blog, this is the web address of the blog post that is the contribution. For a video, it might be you tube video address. etc. This implies that even blog comments should have a unique URI so they can be identified by the protocol.

Each contribution has a domain that maintains the authoritative copy of the contribution. In most cases this would be the domain in the contribution URI, but you could imagine that should a domain disappear overnight a hub or aggregator may become the new authoritative source of a contribution. Domains would have the option of not maintaining a contribution.

Importantly (and probably a defining aspect of the protocol) A contribution has one or more antecedents; contributions it is framed as a reply to. This is the URI(s) of those contributions.

Each contribution has a title and body that is syndicated with the contribution. The syndicated body is rich, since contributions themselves are rich. I would assume it would be in sanitised HTML and standardised microformats.Like the body of current RSS posts.

A contribution has an author. The author should be identified as a URI from a identification provider. I am thinking a scheme such as OpenId would provide the URIs.

Each contribution also has a publication date.

Contributions should be static to maintain their integrity. If you want to change your contribution you had best reply to it. This would be difficult to be police, and like many issues of security and integrity, each contribution domain is either trusted or not - in the same way that email servers, newsgroup services and even individual blogs are mostly trusted today.

So if each contribution includes references to its antecedents, the conversation relevant to that contribution can be traced to its source by examining the antecedent contributions and in turn their antecedents, and so forth.

Discovering subsequent (in the antonym sense to antecedent) contributions on separate domains may require a discovery mechanism, possibly facilitated by registration made by the subsequent contribution made to an off-domain antecedent.

In this sense, conversations should be crawlable.

Contribution domains, hubs and archival centers would host open conversation protocol services. Architecturally contribution domains may be directly federated by other contribution domains or specialist hubs. All support the same open conversation interface. Additionally a conversation service might layer conversation metadata over conversation-less platforms such as Twitter, which are none-the-less potential contribution domains.

Contribution domains are under no obligation to accept any contribution. In fact their value could be by increased by curating contributions to retain quality. However a contribution domain can not prevent a contribution on another domain from referencing their contributions.

The syndicated body of a contribution would have value for search engines but a great deal more value would be obtained by analysing the contribution graph. Page-rank and other analysis would find important parts of the conversation with subsequent and antecedent contributions defining a very accurate context. I suspect a search engine would value higher those contributions that have more subsequent contributions. This would reflect accurately the influence of the contribution on the conversation.

This speculative thought process led me to the services listed at the top of this post.

I have not considered any implementation issues yet. Security, scalability need to be addressed, as do the actual forms of communication and syndication of this information. This is a useful thought experiment, but I do believe that understanding conversation is one of the next challenges of the web.

Let me know if you have any contributions to make to this post.

Innovation in the Postal Service

Australia Post, the government owned postal service operating within Australia, is expanding its business into insurance. Starting with car and later home insurance, Australia post will use its outlets and distribution network to launch its entry into the financial services sector.

The bid to enter new markets is not new for Australia Post, nor for postal services globally. Australian post offices already sell everything from children's books to printers. Other services include electronic bill payment, the taking of passport photos and soon, identity-proving. Australia Post cites other postal services globally that have made a successful entrance into the insurance sector.

With the rise of email and other forms of electronic fulfillment, traditional "snail" mail is no longer paying the bills.

Rather than growing new heads, I wonder if Australia Post and other postal services worldwide should consider innovation within their core business. Like the mainstream news media, music and content publishers, it is difficult to imagine a less proactive and more reactive approach to the opportunities - and the increased, agile competition - posed by the digital world.

Here are some ideas - off the cuff - for innovating in the postal industry in a valuable (and therefore monetisable) way.

Digitise the mail. Postal services largely missed the boat on electronic mail (you have to wonder how) but there is still is still ample opportunity to provide a gateway service to the digital world. Swiss Post has already established a service (Swiss Postbox) that, for a fee, will scan the outside of your mail and provide you with an online interface to determine which items actually get delivered or scanned and delivered electronically. The rest - whether it is junk mail or letters from your local politician- gets usefully recycled.

This is win for all parties. The post company can digitise closer to the source and save distribution costs while it earns a (currently premium) fee in the process. The end user finds out about mail sooner and more securely, and can receive their mail with much greater flexibility in the storage and backup. Do not forget the environment; removing the environmental costs of distribution and redirecting mail in bulk quantities to recycling is beneficial there as well.

Virtualise the Address. I find it astounding to think that to send something to me you need to know my current physical address. I am still receiving mail for occupants of my house that have not lived here for 3 years. Their physical address has changed, but the hundreds of records held by companies about them have obviously not. When I go overseas in a few weeks I will have the same problem with my own mail. It is also a concern for privacy; I may want to allow a business to send mail to me, but do I have to disclose where I physically reside?

A virtual address would resemble a post office box address, but would be delivered not to a physical box but to the physical address on file or an electronic account if the mail service is digital. The virtual address could have a suburb like a post office address - this would form the basis of postage for the sender, with the recipient paying any additional amounts required to ship to a different post or zip code. The recipient could manage their physical address online, changing it for all of their future mail with a few clicks.

Despite the extreme convenience of this feature for consumers, businesses and even the wider economy will benefit as more mail reaches the intended recipient. Virtual addressing may begin at a premium price, but ultimately I believe it should become very low in price or free.

Sell registered email services. Email is in widespread use but is frequently abused. Phishing attacks and spam mail frequently spoof other mail senders and are so common that both mail clients and recipients need more ways the verify the actual sender. Postal services could offer registered email services at a low price (say, 1 cent per email) that use best practices for mail server authentication including reverse DNS servers, SPF records and signing with DKIM or DomainKeys.

Confidentiality in email is likewise hard to achieve with mass emailings, but postal services could also offer encryption gateways (PGP or otherwise) or even host confidential messages for users to access over secure (SSL/TLS) protocols. Such a service could also provide confirmation of the receipt of the message.

Host electronic forms. In Australia at least the semi-local Post Office is also a source and submission point of paper forms for various postal and government services. When written forms are received, they are painstakingly entered into computer systems (sometimes via optical character recognition, but many times manually). For consumers paper form submission often means lunchtime excursions to busy post offices with a long wait in line either to get the form or to submit it.

Electronic forms are not new. Adobe PDF supports them natively (ever wondered why the Adobe Reader installation was so huge?) and many web sites offer form and survey services. The biggest challenge for official forms is authentication - are you who you say you are?

Offering an electronic forms facility - first for postal services, later for other government sectors - would be a key convenience the postal service could offer. Best of all, offering electronic form submission would pay for itself in reducing the man power required at post offices.

These are just a few ideas, but the hopefully they demonstrate that postal services do not need to look outside their core business to find new revenue streams and in the process improve the service they provide to consumers. Rather than being victims of technology a company can choose innovation and instead be a leader of their industry. This requires re-examining many of the assumptions that underly the current business model and asking what the market actually wants.

The postal service is a good example of an industry that has barely changed in the last one hundred years but which will need a shake up to remain relevant in the new digital world. The moves by companies like Swiss Post seem a step in the right direction. There is more that can be done.

News Futures

How will we be consuming news in ten or twenty years? Lately I have been pondering the future of our traditional news institutions, particularly newspapers, and how they will change in the years ahead.

Its an open secret that lately the business model of newspapers has been looking ill - possibly terminal. Readership of hardcopy is declining and once-lucrative advertisers are being wooed by much cheaper advertising online. Some newspapers have gone under completely, leaving towns without a local rag entirely.

The world has changed significantly since the arrival of the internet. The internet has fundamentally altered the economics of media. In what is rapidly becoming the olden days, a town may only have had one or two local papers with broad-based appeal. They would offer a limited amount of advertising space, and existed in an advertising ecosystem where all other highly effectively advertising was similiarly limited. There are only so many hours in the day to show advertisements on television, for example. And from the consumer's perspective, if you wanted more information about the day's news than what was covered in television bulletins, newspapers were the only thing that could provide it.

Now in the internet age, things are different. Advertising space is no longer a scarcity, since the internet offers a nearly boundless market. And if I want detailed information about a news story, I do not need to buy anything. I can Google it. I no longer need to buy a sports section I do not want. I know longer need to pay for horoscopes or tide information or Family Circus or anything else I never have read and never will read. I get it for free, and I get only what I want, and with news feeds and social networking and book marking sites, it now comes to me as well. It has never been a better time to be a news consumer.

But pity the poor newspaper. Newspapers have being trying to recreate their product online using a few different approaches. But they are struggling to turn any sort of profit and are frequently making large losses. The Wall Street Journal and the Financial Times are the only well-known exceptions, and they offer paid content for niche markets that economic advantage from them. Advertising on the online editions has so far failed to recoup the cost of free services.

Just this past week Rupert Murdoch has announced all of the News Corporation online sites would be beginning to charge for content. According to Murdoch, quality journalism costs money, and that consumers should be charged for that journalism:

"Quality journalism is not cheap, and an industry that gives away its content is simply cannibalising its ability to produce good reporting."

"The digital revolution has opened many new and inexpensive distribution channels but it has not made content free. We intend to charge for all our news websites."

Fairfax Media has rapidly closed ranks with their competitor and announced similiar plans.

In a surprisingly churlish piece, The Australian pleaded the case for paid online news content. They attempt to marginalise the 99% of bloggers writing on the subject (their estimate) who had a negative reaction. Correct, those bloggers are not the total market as the Australian points out, but they are the future of the market, so listen up (In this and other stories, you might be forgiven for thinking bloggers were some form of extremist. Not so, they are comprised of increasing numbers of regular people who are increasingly empowered to express their opinion. Ignore them at your peril.)

A number of strategies have been proposed for charging users for news content. A subscription could be bought to cover the newspaper as a whole or specific sections. A metering system might let you into the news site for one or two pages before prompting you for your subscription details. Micro-payments might be used to extract small amounts of money for only the content you want. The Australian article proposes a paid news site with social networking services, highly granular subscription options (including crosswords!) and an online marketplace. It actually sounds pretty awful and redundant to me, but the Australian is correct in implying that innovation may provide a solution in this space (more on this in a later blog post).

Journalism educator and blogger Jeff Jarvis, author of the book 'What Would Google Do', calls the move to erect pay walls around content suicidal. He argues that

Charging for content brings marketing and customer-service costs. Online, it reduces audience and the advertising they justify. Putting content behind a wall cuts it off from search and links; they cut off your Googlejuice.

Jarvis goes on to point out the elephant in the room. If you charge for news content, free competitors will eat you for breakfast. On the internet, your competitor is a click away. The competitors are bloggers, the news wire, government releases and in the vanguard quality publicly owned sites such as the (Australian) ABC in Australia or the BBC in the UK.

My instinct is to agree with Jarvis. For every user you manage to extract money from, you will drive from your site ten others. Your new profits will be swallowed by the marketing and sales effort required to grow the business. You will constantly fighting a battle to get users to come to your site, trial your product and for some reason believe your content is qualitatively better than the free option. And if a majority of news companies suddenly decide now is the time to start charging, will competition quickly erode the planned revenue base?

Are newspapers and the the dinosaurs of our time, doomed to become extinct? What will replace them? Can news companies monetise their existing sites and manage to pull through? Or will they need a radical transformation?

I am not a journalist, an analyst or a news mogul, but I do have a few opinions about these and related matters. Check back here later to see more entries on the subject from me. Comments are very welcome.

Welcome to Doom-and-Gloom.net!

This is a blog I created to chronicle my ramblings about the state of the world. It is not all negative but - and this is a big 'but' - if you have never appreciated the unlikeliness of our planet, our ecosystem, our species, our civilisation and ourselves you might find some of the topic possibly negative.

Enjoy!