The Birth of "The Metaweb" — The Next Big Thing — What We are All Really Building

Originally developed at Netscape, a new technology called RSS has risen from the dead to ignite the next-evolution of the Net. RSS represents the first step in a major new paradigm shift — the birth of “The Metaweb.” The Metaweb is the next evolution of the Web — a new layer of the Web in fact — based on “microcontent.” Microcontent is a new way to publish content that is more granular, modular and portable than traditional content such as files, Web pages, data records, etc.

On the existing Web, information is typically published in large chunks — “sites” comprised of “pages.” In the coming microcontent-driven Metaweb, information will be published in discrete, semantically defined “postings” that can represent an entire site, a page, a part of a page, or an individual idea, picture, file, message, fact, opinion, note, data record, or comment.

Metaweb postings can be hosted like Web pages in particular places and/or they can be shipped around the Net using RSS in a publish-subscribe manner. Webloggers for example create microcontent every time they post to their blogs. Each blog posting is a piece of microcontent. End-users can subscribe to get particular pieces of microcontent they are interested in by signing up to track “RSS channels” using “RSS Readers” that poll those channels periodically for new pieces of microcontent.

RSS resembles traditional “publish and subscribe” except that it scales to the entire Internet and is based on new XML open standards. Unlike “push technology” RSS and the microcontent model is based instead on “pull” — just like the Web itself — RSS Readers periodically poll sources for new RSS content and pull it down instead of having it pushed at them. Thus, unlike push technology, with RSS the control is in the hands of opt-in end-users. These differences, combined with RSS’s use of open HTTP protocols and XML/RDF formats have led to rapid adoption and viral spread of RSS technologies — principally within the Weblogging and information services communities. But that’s about to change.

RSS is poised to become The Next Big Thing. There are many reasons for this — for one thing, e-mail is no longer useful as a content distribution, alerting and marketing medium. E-Mail’s rapidly eroding signal-to-noise ratio is leading content providers and end-users to seek alternative, more mutually-effective avenues for interacting with one another. Another force that is driving RSS adoption is the rise of Weblogging.

My projections indicate that within 5 years almost every Weblog will provide an RSS channel of its content. In coming years a large percentage of consumers and professionals are expected to begin blogging — Weblogs are the new homepages; everyone should have one.

Within 5 years, if RSS grows as I expect, we will see it supplant e-mail as the primary alerting and marketing channel for “B2C” communications. To put it simply, businesses and their customers both benefit from interacting via RSS instead of e-mail for “1-way” interactions such as content publishing, notifications, etc. Based on that, I predict that every medium to large corporate Web site and every major publication and wire service, as well as an increasing number of enterprise applications and services will publish and subscribe to numerous RSS channels. Already we see the beginning of this with numerous major organizations embracing RSS from IBM, Microsoft and Sun to The New York Times, ABC News and WIRED to name a few examples.

So, 30 million bloggers at 1 feed each + 2 milllion small, medium and large businesses at an average of 20 feeds each + 2 million web sites and information services providers at an average of 10 feeds each + 10 major portals and online services at an average of 1,000,000 feeds each + 100 million desktop and enterprise applications producing 1 feed each …. you can see where this is headed. To be conservative let’s assume that the numbers turn out to be less than what I project — that is still 50 million to 100 million feeds online within 5 years. And that’s a growth curve that looks a lot like the first wave of the Web. Just as everyone “had to have” an e-mail account and a Web page, they will also soon need and want to have an RSS reader and their own RSS channel. That’s a big opportunity.

But RSS is just the first step in the evolution of the Metaweb. The next step will be the Semantic Web. RSS begins the process of getting end-users and content providers to use metadata. The next step is to make that metadata more interoperable, more understandable, more useful. This takes place using ontologies and emerging tools for working with “semantic metadata” — metadata for which formally defined semantics exists. Just providing metadata is not enough — the meaning of that metadata has to be defined somewhere in a formal, rigorous, manner that computers can understand automatically. The Semantic Web transforms data and metadata from “dumb data” to “smart data.” When I say “smart data” I mean data that carries increased amounts of information about its own meaning, structure, purpose, context, policies, etc. The data is “smart” because the knowledge about the data moves with the data, instead of being locked in an application. So the Semantic Web is a web of “smart data” — a Web of semantically defined metadata. The Semantic Web is already evolving naturally from the emerging confluence of Blogs, Wikis, RSS feeds, RDF tools, ontology languages such as OWL, rich ontologies, inferencing engines, triplestores, and a growing range of new tools and services for working with metadata. But the key is that we don’t have to wait for the Semantic Web for metadata to be useful. The Metaweb is already happening. RSS is already useful and it’s happening now.

As I write this on the leading edge of 2004 — a little more than ten years after the Web began — I am aware that we are witnessing the birth of the next generation of the Net. I remember watching the birth of the HTML-Web as a technology analyst/editor at Individual, Inc in the early 90’s. My job was to manage a collection of intelligent agents that scanned hundreds of newswires and content archives to produce filtered strategic newsfeeds for major customers. My beat was “emerging technologies” — every night I had to Q-A the output of my agents by reading around 1400 articles and press releases about new technologies in a 4 hour period.

It was in the midst of that firehose of information that I noticed the birth of HTML and HTTP, the rise of early hypertext systems, the first browsers — and I realized that “something big” was afoot. At the beginning the pattern wasn’t evident from reading individual articles — only by reading 1400 articles a night could one see the early meme-signatures of the HTML-Web flashing across hundreds of media outlets like a sequence of blinking Christmas-tree lights. That recognition led me to leave Individual and co-found EarthWeb in 1994 — because I wanted to be a part of building the Web, not just watching it! Today, just like in 1994 with HTML, it is much the same situation and again I am back to building again — Radar Networks, our stealth venture, is developing a new platform for the Metaweb that will open up a range of new capabilities for sharing metadata.

The baby Metaweb has already been born, but so far only the early-adopters and Web-veterans have noticed it. To those who “were there” the first time around there is a recognizable feeling of momentum — of “something big” happening again. It’s going to be a fun ride!

Notes:

1. A new syndication format based on RSS is being proposed as an open standard. Called Atom it promises to provide a vendor neutral, extensible format for weblogging.

2. Why the term “Metaweb”? A reader suggested that the prefix “Meta” was too technical for consumers. I don’t think so however — after all they use the term “Internet” without any problem and that is not exactly a consumer-friendly word when you think about its meaning and origin. The concept of the Metaweb is that it is a new layer of the existing Web, that’s why the name should really contain “Web” in it.

3. Here is a good timeline on the history and origins of RSS