Peace in the Middle East: Could Alternative Energy Be the Solution?

I have been thinking about the situation in the Middle East and also the rise of oil prices, peak oil, and the problem of a world economy based on energy scarcity rather than abundance. There is, I believe, a way to solve the problems in the Middle East, and the energy problems facing the world, at the same time. But it requires thinking “outside the box.”

Middle Eastern nations must take the lead in freeing the world from dependence on their oil. This is not only their best strategy for the future of their nations and their people, but also it is what will ultimately be best for the region and the whole world.

It is inevitable that someone is going to invent a new technology that frees the world from dependence on fossil fuels. When that happens all oil empires will suddenly collapse. Far-sighted, visionary leaders in oil-producing nations must ensure that their nations are in position to lead the coming non-fossil-fuel energy revolution. This is the wisdom of “cannibalize yourself before someone else does.”

Middle Eastern nations should invest more heavily than any other nations in inventing and supplying new alternative energy technologies. For example: hydrogen, solar, biofuels, zero point energy, magnetic power, and the many new emerging alternatives to fossil fuels. This is a huge opportunity for the Middle East not only for economic reasons, but also because it may just be the key to bringing about long-term sustainable peace in the region.

There is a finite supply of oil in the Middle East — the game will and must eventually end. Are Middle Eastern nations thinking far enough ahead about this or not? There is a tremendous opportunity for them if they can take the initiative on this front and there is an equally tremendous risk if they do not. If they do not have a major stake in whatever comes after fossil fuels, they will be left with nothing when whatever is next inevitably happens (which might be very soon).

Any Middle Eastern leader who is not thinking very seriously about this issue right now is selling their people short. I sincerely advise them to make this a major focus going forward. Not only will this help them to improve quality of life for their people now and in the future, but it is the best way to help bring about world peace. The Middle East has the potential to lead a huge and lucrative global energy Renaissance. All it takes is vision and courage to push the frontier and to think outside of the box.

Continue reading

My Burma Meme Spreads to 17,000 Web Pages in just one week!

I’ve been tracking the progress of my Burma protest meme. In just under one week it has spread to almost 17,000 web pages and it continues to grow. (For the latest number, click here). It’s great to see the blogosphere pick this up, and I’m glad to be able to do something to help raise awareness of this important human rights issue.

This meme is also an example of an interesting new way to spread content on the Web — whether for a protest or an ad or any other kind of announcement. It’s kind of like a chain letter, but via weblogs. There are many different ways to structure these memes with varying levels of virality and benefit to participants. For some earlier work I’ve done on meme propagation on the Web see my GoMeme experiments from a few years ago. In those experiments I created a series of memes that spread widely through the blogosphere, based on different viral messages, surveys, and benefits to participants. Other people then tracked the statistics of the memes as they spread. It turned out to be a very interesting study of superdistribution of content along social networks.

Burma Update: Protestors Cremated Alive; Monks Massacred in Jungle

The situation in Burma is far worse than the mainstream media has reported so far. Watch this video that was just smuggled out showing soldiers beating unarmed protesters. There are now reports coming in from eyewitnesses of young school students being shot by the army, masses of injured protestors being cremated alive, and thousands of monks and other protesters being killed and dumped in mass graves in the jungles. The junta has now imprisoned thousands of monks and plans to "send them away." The Burmese military junta is known for torture, summary executions, and is listed with Somalia as one of the most corrupt regimes in the world. This is essentially a form of genocide or state-sponsored terrorism — in this case by a regime against its own people. Help get the word out. Sign this petition. This has to be stopped. The Burmese people are helpless and need protection from the world community.

Help the People of Burma — Post this Meme on Your Blog!

Note: This is a new kind of online protest that uses blogs to spread a petition globally. To participate, just add your blog by following the instructions in this blog post.

This not an issue of partisan politics, this is an issue of basic human rights and democracy. Please help to prevent a human tragedy in Burma by adding your blog and asking others to do the same.

By passing this meme on through the blogosphere hopefully we can generate more awareness and avert a serious tragedy. As concerned world-citizens this something we bloggers can do to help.

How to participate:

1. Copy this entire post to your blog, including this special number: 1081081081234

2. After a few days, you can search Google for the number 1081081081234 to find all blogs that are participating in this protest and petition. Note: Google indexes blogs at different rates, so it could take longer for your blog to show up in the results. Also note — if you want an accurate count of the number of blogs that joined this petition, make sure you scroll to the bottom of the Google results and click to view results that were omitted because they were similar.

3. If you know how to add tags to your blog posts, add the Technorati tag 1081081081234 to your post as well. This will make your post findable sooner in Technorati.

THE SITUATION IN BURMA AND WHY IT MATTERS TO ALL OF US

There is no press freedom in Burma and the government has started turning off the Internet and other means of communication, so it is difficult to get news out. Individuals on the ground have been sending their day-by-day reports to the BBC, and they are heartbreaking. I encourage you to read these accounts to see for yourself what is really going on in Burma. Please include this link in your own blog post.

The situation in Burma is increasingly dangerous. Hundreds of thousands of unarmed peaceful protesters, including monks and nuns, are risking their lives to march for democracy against an unpopular but well-armed military dictatorship that will stop at nothing to continue its repressive rule. While the generals in power and their families are literally dripping in gold and diamonds, the people of Burma are impoverished, deprived of basic human rights, cut off from the rest of the world, and increasingly under threat of violence.

This week the people of Burma have risen up collectively in the largest public demonstrations against the ruling Junta in decades. It’s an amazing show of bravery, decency, and democracy in action. But although these protests are peaceful, the military rulers are starting to crack down with violence. Already there have been at least several reported deaths, and hundreds of critical injuries from soldiers beating unarmed civilians to the point of death.

The actual fatalities and injuries are probably far worse, but the only news we have is coming from individuals who are sneaking reports past the authorities. Unfortunately it looks like a large-scale blood-bath may ensue — and the victims will be mostly women, children, the elderly and unarmed monks and nuns.

Contrary to what the Burmese, Chinese and Russian governments have stated, this is not merely a local internal political issue, it is an issue of global importance and it affects the global community. As concerned citizens, we cannot allow any government anywhere in the world to use its military to attack and kill peacefully demonstrating, unarmed citizens.

In this modern day and age violence against unarmed civilians is unacceptable and if it is allowed to happen, without serious consequences for the perpetrators, it creates a precedent for it to happen again somewhere else.
If we want a more peaceful world, it is up to each of us to make a personal stand on these fundamental issues whenever they arise.

Please join me in calling on the Burmese government to negotiate peacefully with its citizens, and on China to intervene to prevent further violence. And please help to raise awareness of the developing situation in Burma so that hopefully we can avert a large-scale human disaster there.

Dangerous Situation in Burma Developing

The nation of Myanamar (Burma) is currently ruled by a repressive and corrupt military dictatorship. Past attempts to bring about democracy have met with violent crackdowns by the government, usually against unarmed civilians. In one past incident, approximately 3000 protesters were killed by the military.

Now a new situation is developing that could rapidly lead to mass civilian casualties, if the international community does not intervene in time. Led by thousands of Buddhist monks, approximately 100,000 Burmese civilians marched to demonstrate their support for democracy in Burma.

So far the military has not reacted with violence, but there are growing warning signs that this may happen soon. The international community, and all people who believe in freedom, should stand up and support the brave citizens of Burma who are putting their lives at risk for the principle of democracy, a principle which unfortunately is eroding worldwide according to a recent study.

Chinese Authorities Make it Illegal to Reincarnate Without a Permit

This just in. The Chinese Government, in their ongoing campaign against the Dalai Lama and Buddhism in Tibet, have announced a new law making it illegal for a Buddha to reincarnate without a state permit. This law is designed effectively to put an end to the form of Buddhism practiced in Tibet within one generation. The Tibetan monastic system is organized around reincarnated "tulkus" who are usually identified though a battery of tests at a young age and are then tutored and groomed to run monasteries and monastic schools when they grow up. This new law  will make it a crime for anyone to be identified as a reincarnated lama without state approval — meaning that from now on the Chinese government will be in a position to appoint or deny whoever they select to lead the Buddhists of Tibet. Of course the irony is that the Chinese state is atheistic and in making this law they seem to be contradicting themselves in admitting that Buddhas exist and can reincarnate. But furthermore, the notion that the Chinese state is qualified to identify or approve reincarnated Buddhas is patently absurd. This is one more unfortunate step in China’s 50-year cultural genocide of the Tibetan people.

Congratulations to Adam Cheyer and Change.org

My friend and colleague, Adam Cheyer, from SRI, recently helped to advise and launch Change.org — a social network for nonprofit activism. It’s a great site for a great cause. It’s also a great example of a special-purpose social network — a useful social network. Congratulations to Adam and the team over there! Read the TechCrunch review here.

Almost Unbelievably Stupid — Diebold Posts Photo of Voting Machine Key — Hackers Replicate Key From Photo

This one really takes the cake. Diebold, the makers of the infamously hackable electronic voting machines, has done it again. They posted photos for replacement keys for their voting machines on their Web site. Unfortunately the photos were high enough resolution that anyone who knows how to make keys could easily copy them. And that’s exactly what one hacker did and he documented how he did it too — using just the photo and a file he made a set of universal keys that can unlock any Diebold voting machine. DUH! So now this basically means that the security of every Diebold voting machine is potentially compromised. The only way for Diebold to fix this would be to replace all the locks in all their machines. Will they do this? Highly unlikely. More on this story here.

A Passenger's Bill of Rights — Sign me Up!

Where do I sign up to support this initiative? A group of exasperated US airline passengers who were stuck on the tarmac for 10 hours last month have come up with a proposal for a bill to establish a Passenger’s Bill of Rights, which would include the following provisions:

Establishing procedures for airlines to return passengers to a terminal gate after three hours on the tarmac.

Requiring airlines to respond to complaints within 24 hours and resolve them within two weeks.

Forcing airlines to publish a list of chronically delayed flights online.

Compensation for bumped passengers or passengers whose flights are
delayed by more than 12 hours at 150 percent of the ticket price.

Compensation for passengers whose baggage is lost or mishandled.

Creation of a Passenger Review Committee made up of nonairline consumers to review and investigate complaints.

Having been in situations like this myself I can attest to the fact that something has to be done about this. Last summer I was stuck on the tarmac in Texas in 120 degree heat. The plane’s air conditioner was turned off because they wanted to conserve fuel, so inside the plane it was probably 130 degrees. To top that off nobody could get up, and the crew had to remain seated as well so they couldn’t even bring people water. This lasted for 2 hours. I was almost passing out — I can only imagine what it must have been like for children, the elderly or people with medical conditions. Airlines treat passengers only a little better than baggage, and that is not saying much! Something needs to be done about this and I’m glad an initiative is forming to take action. Count me in!

Minding The Planet — The Meaning and Future of the Semantic Web

NOTES

Prelude

Many years ago, in the late 1980s, while I was still a college student, I visited my late grandfather, Peter F. Drucker, at his home in Claremont, California. He lived near the campus of Claremont College where he was a professor emeritus. On that particular day, I handed him a manuscript of a book I was trying to write, entitled, “Minding the Planet” about how the Internet would enable the evolution of higher forms of collective intelligence.

My grandfather read my manuscript and later that afternoon we sat together on the outside back porch and he said to me, “One thing is certain: Someday, you will write this book.” We both knew that the manuscript I had handed him was not that book, a fact that was later verified when I tried to get it published. I gave up for a while and focused on college, where I was studying philosophy with a focus on artificial intelligence. And soon I started working in the fields of artificial intelligence and supercomputing at companies like Kurzweil, Thinking Machines, and Individual.

A few years later, I co-founded one of the early Web companies, EarthWeb, where among other things we built many of the first large commercial Websites and later helped to pioneer Java by creating several large knowledge-sharing communities for software developers. Along the way I continued to think about collective intelligence. EarthWeb and the first wave of the Web came and went. But this interest and vision continued to grow. In 2000 I started researching the necessary technologies to begin building a more intelligent Web. And eventually that led me to start my present company, Radar Networks, where we are now focused on enabling the next-generation of collective intelligence on the Web, using the new technologies of the Semantic Web.

But ever since that day on the porch with my grandfather, I remembered what he said: “Someday, you will write this book.” I’ve tried many times since then to write it. But it never came out the way I had hoped. So I tried again. Eventually I let go of the book form and created this weblog instead. And as many of my readers know, I’ve continued to write here about my observations and evolving understanding of this idea over the years. This article is my latest installment, and I think it’s the first one that meets my own standards for what I really wanted to communicate. And so I dedicate this article to my grandfather, who inspired me to keep writing this, and who gave me his prediction that I would one day complete it.

This is an article about a new generation of technology that is sometimes called the Semantic Web, and which could also be called the Intelligent Web, or the global mind. But what is the Semantic Web, and why does it matter, and how does it enable collective intelligence? And where is this all headed? And what is the long-term far future going to be like? Is the global mind just science-fiction? Will a world that has a global mind be good place to live in, or will it be some kind of technological nightmare?

I’ve often joked that it is ironic that a term that contains theword “semantic” has such an ambiguous meaning for most people. Mostpeople just have no idea what this means, they have no context for it,it is not connected to their experience and knowledge. This is aproblem that people who are deeply immersed in the trenches of theSemantic Web have not been able to solve adequately — they have notfound the words to communicate what they can clearly see, what they areworking on, and why it matters for everyone. In this article I havetried, and hopefully succeeded, in providing a detailed introductionand context for the Semantic Web fornon-technical people. But even technical people working in the fieldmay find something of interest here as I piece together the fragmentsinto a Big Picture and a vision for what might be called “Semantic Web2.0.”

I hope the reader will bear with me as Ibounce around across different scales of technology and time, and fromthe extremes of core technology to wild speculation in order to tellthis story. If you are looking for the cold hardscience of it all, this article will provide an understanding but willnot satisfy your need for seeing the actual code; there are otherplaceswhere you can find that level of detail and rigor. But if you want tounderstand what it all really means and what the opportunity and futurelookslike – this may be what you are looking for.

I should also note that all of this is my personal view of what I’vebeen working on,and what it really means to me. It is not necessarily the official viewof the mainstream academic Semantic Web community — although there arecertainly many places where we all agree. But I’m sure that somereaders will certainly disagree or raise objections to some of myassertions, and certainly to my many far-flung speculations about thefuture. I welcome those different perspectives; we’re all trying tomake sense of this and the more of us who do that together, the more wecan collectively start to really understand it. So please feel free towrite your own vision or response, and please let me know so I can linkto it!

So with this Prelude in mind, let’s get started…

The Semantic Web Vision

The Semantic Web is a set of technologies which are designed toenable aparticular vision for the future of the Web – a future in which allknowledge exists on the Web in a format that software applications canunderstand andreason about. By making knowledge more accessible to software, softwarewillessentially become able to understand knowledge, think about knowledge,and createnew knowledge. In other words, software will be able to be moreintelligent –not as intelligent as humans perhaps, but more intelligent than say,your wordprocessor is today.

The dream of making software more intelligent has been around almost as longas software itself. And although it is taking longer to materialize than past experts hadpredicted, progress towards this goal is being steadilymade. At the same time, the shape of this dream is changing. It is becomingmore realistic and pragmatic. The original dream of artificial intelligence wasthat we would all have personal robot assistants doing all the work we don’twant to do for us. That is not the dream of the Semantic Web. Instead, today’sSemantic Web is about facilitating what humans do – it is about helping humansdo things more intelligently. It’s not a vision in which humans do nothing andsoftware does everything.

The Semantic Web vision is not just about helping software become smarter –it is about providing new technologies that enable people, groups,organizations and communities to be smarter.

For example, by providing individuals with tools that learn about what theyknow, and what they want, search can be much more accurate and productive.

Using software that is able to understand and automatically organize largecollections of knowledge, groups, organizations and communities can reachhigher levels of collective intelligence and they can cope with volumes ofinformation that are just too great for individuals or even groups tocomprehend on their own.

Another example: more efficient marketplaces can be enabled by software thatlearns about products, services, vendors, transactions and market trends andunderstands how to connect them together in optimal ways.

In short, the Semantic Web aims to make software smarter, not just for itsown sake, but in order to help make people, and groups of people, smarter. Inthe original Semantic Web vision this fact was under-emphasized, leading to theimpression that Semantic Web was only about automating the world. In fact, it isreally about facilitating the world.

The Semantic Web Opportunity

The Semantic Web is one of the most significant things to happen since theWeb itself. But it will not appear overnight. It will take decades. It willgrow in a bottom-up, grassroots, emergent, community-driven manner just likethe Web itself. Many things have to converge for this trend to really take off.

The core open standards already exist, but the necessary development tools haveto mature, the ontologies that define human knowledge have to come into beingand mature, and most importantly we need a few real “killer apps” to prove thevalue and drive adoption of the Semantic Web paradigm. The first generation ofthe Web had its Mozilla, Netscape, Internet Explorer, and Apache – and it alsohad HTML, HTTP, a bunch of good development tools, and a few killer apps andservices such as Yahoo! and thousands of popular Web sites. The same things arenecessary for the Semantic Web to take off.

And this is where we are today – this all just about to start emerging.There are several companies racing to get this technology, or applications ofit, to market in various forms. Within a year or two you will see mass-consumerSemantic Web products and services hit the market, and within 5 years therewill be at least a few “killer apps” of the Semantic Web. Ten years from nowthe Semantic Web will have spread into many of the most popular sites andapplications on the Web. Within 20 years all content and applications on theInternet will be integrated with the Semantic Web. This is a sea-change. A bigevolutionary step for the Web.

The Semantic Web is an opportunity to redefine, or perhaps to better define,all the content and applications on the Web. That’s a big opportunity. Andwithin it there are many business opportunities and a lot of money to be made. It’snot unlike the opportunity of the first generation of the Web. There areplatform opportunities, content opportunities, commerce opportunities, searchopportunities, community and social networking opportunities, and collaborationopportunities in this space. There is room for a lot of players to compete andat this point the field is wide open.

The Semantic Web is a blue ocean waiting to be explored. And like anyunexplored ocean its also has its share of reefs, pirate islands, hidden treasure, shoals,whirlpools, sea monsters and typhoons. But there are new worlds out there to be discovered,and they exert an irresistible pull on the imagination. This is an excitingfrontier – and also one fraught with hard technical and social challenges thathave yet to be solved. For early ventures in the Semantic Web arena, it’s notgoing to be easy, but the intellectual and technological challenges, and the potentialfinancial rewards, glory, and benefit to society, are worth the effort andrisk. And this is what all great technological revolutions are made of.

Semantic Web 2.0

Some people who have heard the term “Semantic Web” thrown around too muchmay think it is a buzzword, and they are right. But it is not just a buzzword –it actually has some substance behind it. That substance hasn’t emerged yet,but it will. Early critiques of the Semantic Web were right – the early visiondid not leverage concepts such as folksonomy and user-contributed content atall. But that is largely because when the Semantic Web was originally conceivedof Web 2.0 hadn’t happened yet. The early experiments that came out of researchlabs were geeky, to put it lightly, and impractical, but they are already beingfollowed up by more pragmatic, user-friendly approaches.

Today’s Semantic Web – what we might call “Semantic Web 2.0” is a kinder,gentler, more social Semantic Web. It combines the best of the original visionwith what we have all learned about social software and community in the last10 years. Although much of this is still in the lab, it is already starting totrickle out. For example, recently Yahoo! started a pilot of the Semantic Webbehind their food vertical. Other organizations are experimenting with usingSemantic Web technology in parts of their applications, or to store or mapdata. But that’s just the beginning.

The Google Factor

Entrepreneurs, venture capitalists and technologists are increasinglystarting to see these opportunities. Who will be the “Google of the SemanticWeb?” – will it be Google itself? That’s doubtful. Like any entrenchedincumbent, Google is heavily tied to a particular technology and worldview. Andin Google’s case it is anything but semantic today. It would be easier for anupstart to take this position than for Google to port their entireinfrastructure and worldview to a Semantic Web way of thinking.

If it is goingto be Google it will most likely be by acquisition rather than by internal origination. Andthis makes more sense anyway – for Google is in a position where they can just wait and buy the winner,at almost any price, rather than competing in the playing field. One thing to note however is that Google has at least one product offering that shows some potential for becoming a key part of the Semantic Web. I am speaking of Google Base, Google’s open database which is meant to be a registry for structured data so that it can be found in Google search. But Google Base does not conform to or make use of the many open standards of the Semantic Web community. That may or may not be a good thing, depending on your perspective.

Of course the downside of Google waiting to join the mainstream Semantic Web community until after the winner is announced is very large – once there is a winner it may be too late for Google to beat them. Thewinner of the Semantic Web race could very well unseat Google. The strategistsat Google are probably not yet aware of this but as soon as they seesignificant traction around a major Semantic Web play it will become of interestto them.

In any case, I think there won’t be just one winner, there will be severalmajor Semantic Web companies in the future, focusing on different parts of theopportunity. And you can be sure that if Google gets into the game, every majorportal will need to get into this space at some point or risk becomingirrelevant. There will be demand and many acquisitions. In many ways the Semantic Web will not be controlled by just one company — it will be more like a fabric that connects them all together.

Context is King — The Nature ofKnowledge

It should be clear by now that the Semantic Web is all about enablingsoftware (and people) to work with knowledge more intelligently. But what isknowledge? Knowledge is not just information. It is meaningful information – itis information plus context. For example, if I simply say the word “sem” toyou, it is just raw information, it is not knowledge. It probably has nomeaning to you other than a particular set of letters that you recognize and asound you can pronounce, and the mere fact that this information was stated byme.

But if I tell you that “sem” it is the Tibetan word for “mind” then suddenly,“sem means mind in Tibetan” to you. If I further tell you that Tibetans have about as many words for “mind” as Eskimos have for “snow,” this is further meaning. Thisis context, in other words, knowledge, about the sound “sem.” The sound is raw information. When it is given context itbecomes a word, a word that has meaning, a word that is connected to conceptsin your mind – it becomes knowledge. By connecting raw information to context,knowledge is formed.

Once you have acquired a piece of knowledge such as “sem means mind in Tibetan,” you may then also form further knowledgeabout it. For example, you may form the memory, “Nova said that ‘sem means mind in Tibetan.’” You mightalso connect the word “sem” to networks of further concepts you have about Tibet and your understanding of what the word “mind” means.

The mind is the organ of meaning – mind is where meaning is stored,interpreted and created. Meaning is not “out there” in the world, it is purelysubjective, it is purely mental. Meaning is almost equivalent to mind in fact.For the two never occur separately. Each of our individual minds has some way of internally representing meaning — when we read or hear a word that we know, our minds connect that to a network of concepts about it and at that moment it means something to us.

Digging deeper, if you are really curious,or you happen to know Greek, you may also find that a similar sound occurs inthe Greek word, sēmantikós – which means “having meaning” and in turn is the root of the English word “semantic”which means “pertaining to or arising from meaning.” That’s an odd coincidence!“Sem” occurs in Tibetan word for mind, and the English and Greek words that allrelate to the concepts of “meaning” and “mind.” Even stranger is that not only do these words have a similar sound, they have a similar meaning.

With all this knowledge at yourdisposal, when you then see the term “Semantic Web” you may be able to inferthat it has something to do with adding “meaning” to the Web. However, if youwere a Tibetan, perhaps you might instead think the term had something to dowith adding “mind” to the Web. In either case you would be right!

Discovering New Connections

We’ve discovered a new connection — namely that there is an implicit connectionbetween “sem” in Greek, English and Tibetan: they all relate to meaning andmind. It’s not a direct, explicit connection – it’s not evident unless you digfor it. But it’s a useful tidbit of knowledge once it’s found. Unlike the direct migration of the sound “sem” from Greek to English,there may not have ever been a direct transfer of this sound from Greek toSanskrit to Tibetan. But in a strange and unexpected way, they are all connected. This connectionwasn’t necessarily explicitly stated by anyone before, but was uncovered byexploring our network of concepts and making inferences.

The sequence of thought about “sem”above is quite similar to kind of intellectual reasoning and discovery that theactual Semantic Web seeks to enable software to do automatically.  How is this kind of reasoning and discovery enabled? The Semantic Web providesa set of technologies for formally defining the context of information. Just asthe Web relies on a standard formal specification for “marking up” informationwith formatting codes that enable any applications that understand those codesto format the information in the same way, the Semantic Web relies on newstandards for “marking up” information with statements about its context – itsmeaning – that enable any applications to understand, and reason about, the meaning of those statements in the same way.

By applying semantic reasoning agents to large collections of semantically enhanced content, all sorts of new connections may be inferred, leading to new knowledge, unexpected discoveries and useful additional context around content. This kind of reasoning and discovery is already taking place in fields from drug discovery and medical research, to homeland security and intelligence. The Semantic Web is not the only way to do this — but it certainly will improve the process dramatically. And of course, with this improvement will come new questions about how to assess and explain how various inferences were made, and how to protect privacy as our inferencing capabilities begin to extend across ever more sources of public and private data. I don’t have the answers to these questions, but others are working on them and I have confidence that solutions will be arrived at over time.

Smart Data

By marking up information with metadata that formally codifies its context, we can make the data itself “smarter.” The data becomes self-describing. When you get a piece of data you also get the necessary metadata for understanding it. For example, if I sent you a document containing the word “sem” in it, I could add markup around that word indicating that it is the word for “mind” in the Tibetan language.

Similarly, a document containing mentions of “Radar Networks” could contain metadata indicating that “Radar Networks” is an Internet company, not a product or a type of radar technology. A document about a person could contain semantic markup indicating that they are residents of a certain city, experts on Italian cooking, and members of a certain profession. All of this could be encoded as metadata in a form that software could easily understand. The data carries more information about its own meaning.

The alternative to smart data would be for software to actually read and understand natural language as well as humans. But that’s really hard. To correctly interpret raw natural language, software would have to be developed that knew as much as a human being. But think about how much teaching and learning is required to raise a human being to the point where they can read at an adult level. It is likely that similar training would be necessary to build software that could do that. So far that goal has not been achieved, although some attempts have been made. While decent progress in natural language understanding has been made, most software that can do this is limited around particular vertical domains, and it’s brittle — it doesn’t do a good job of making sense of terms and forms of speech that it wasn’t trained to parse and make sense of.

Instead of trying to make software a million times smarter than it is today, it is much easier to just encode more metadata about what our information means. That turns out to be less work in the end. And there’s an added benefit to this approach — the meaning exists with the data and travels with it. It is independent of any one software program — all software can access it. And because the meaning of information is stored with the information itself, rather than in the software, the software doesn’t have to be enormous to be smart. It just has to know the basic language for interpreting the semantic metadata it finds on the information it works with.

Smart data enables relatively dumb software to be smarter with less work. That’s an immediate benefit. And in the long-term as software actually gets smarter, smart data will make it easier for it to start learning and exploring on its own. So it’s a win-win approach. Start with by adding semantic metadata to data, end up with smarter software.

Making Statements About the World

Metadata comes down to making statements about the world in a manner that machines, and perhaps even humans, can understand unambiguously. The same piece of metadata should be interpreted in the same way by different applications and readers.

There are many kinds of statementsthat can be made about information to provide it with context. For example, youcan state a definition such as “person” means “a human being or a legalentity.” You can state an assertion such as “Sue is a human being.” You canstate a rule such that “if x is a human being, then x is a person.”

From thesestatements it can then be inferred that “Sue is a person.” This inference is soobvious to you and me that it seems trivial, but most software today cannot dothis. It doesn’t know what a person is, let alone what a name is. But ifsoftware could do this, then it could for example, automatically organizedocuments by the people they are related to, or discover connections betweenpeople who were mentioned in a set of documents, or it could find documentsabout people who were related to particular topics, or it could give you a listof all the people mentioned in a set of documents, or all the documents relatedto a person.

Of course this is a very basicexample. But imagine if your software didn’t just know about people – it knewabout most of the common concepts that occur in your life. Your software wouldthen be able to help you work with your documents just about as intelligentlyas you are able to do by yourself, or perhaps even more intelligently, becauseyou are just one person and you have limited time and energy but your softwarecould work all the time, and in parallel, to help you.

Examples and Benefits

How could the existence of the Semantic Web and all the semantic metadata that defines it be really useful toeveryone in the near-term?

Well, for example, the problem of email spam would finally be cured:your software would be able to look at a message and know whether it wasmeaningful and/or relevant to you or not.

Similarly, you would never have to file anything by hand again. Your software could atuomate all filing and information organization tasks for you because it would understand your information and your interests. It would be able to figure out when to file something in a single folder, multiple folders, or new ones. It would organize everything — documents, photos, contacts, bookmarks, notes, products, music, video, data records — and it would do it even better and more consistently than you could on your own. Your software wouldn’t just organize stuff, it would turn it into knowledge by connecting it to more context. It could this not just for individuals, but for groups, organizations and entire communities.

Another example: search would bevastly better: you could search conversationally by typing in everyday naturallanguage and you would get precisely what you asked for, or even what youneeded but didn’t know how to ask for correctly, and nothing else. Your searchengine could even ask you questions to help you narrow what you want. You wouldfinally be able to converse with software in ordinary speech and it would understandyou.

The process of discovery would be easier too. You could have software agent that worked as your personal recommendation agent. It would constantly be looking in all the places you read or participate in for things that are relevant to your past, present and potential future interests and needs. It could then alert you in a contextually sensitive way, knowing how to reach you and how urgently to mark things. As you gave it feedback it could learn and do a better job over time.

Going even further with this,semantically-aware software – software that is aware of context, software thatunderstands knowledge – isn’t just for helping you with your information, itcan also help to enrich and facilitate, and even partially automate, yourcommunication and commerce (when you want it to). So for example, your software could help you with your email. It would be able to recommend responses to messages for you, or automate the process. It would be able to enrich your messaging anddiscussions by automatically cross-linking what you are speaking about withrelated messages, discussions, documents, Web sites, subject categories,people, organizations, places, events, etc.

Shopping and marketplaces wouldalso become better – you could search precisely for any kind of product, withany specific attributes, and find it anywhere on the Web, in any store. You could post classified ads and automatically get relevant matches according to your priorities, from all over the Web, or only from specific places and parties that match your criteria for who you trust. You could also easily invent a new custom datastructure for posting classified ads for a new kind of product or service and publishit to the Web in a format that other Web services and applications couldimmediately mine and index without having to necessarily integrate with yoursoftware or data schema directly.

You could publish an entiredatabase to the Web and other applications and services could immediately startto integrate your data with their data, without having to migrate your schemaor their own. You could merge data from different data sources together to create new data sources without having to ever touch or look at an actual database schema.

Bumps on the Road

The above examples illustrate thepotential of the Semantic Web today, but the reality on the ground is that the technology isstill in the early phases of evolution. Even for experienced software engineersand Web developers, it is difficult to apply in practice. The main obstaclesare twofold:

(1) The Tools Problem:

There are very few commercial-gradetools for doing anything with the Semantic Web today – Most of the tools forbuilding semantically-aware applications, or for adding semantics toinformation are still in the research phase and were designed for expertcomputer scientists who specialize in knowledge representation, artificialintelligence, and machine learning.

These tools require a largelearning curve to work with and they don’t generally support large-scaleapplications – they were designed mainly to test theories and frameworks, notto actually apply them. But if the Semantic Web is ever going to becomemainstream, it has to be made easier to apply – it has to be made moreproductive and accessible for ordinary software and content developers.

Fortunately, the tools problem isalready on the verge of being solved. Companies such as my own venture, RadarNetworks, are developing the next generation of tools for building Semantic Webapplications and Semantic Web sites. These tools will hide most of thecomplexity, enabling ordinary mortals to build applications and content thatleverage the power of semantics without needing PhD’s in knowledge representation.

(2) The Ontology Problem:

The Semantic Web providesframeworks for defining systems of formally defined concepts called “ontologies,”that can then be used to connect information to context in an unambiguous way. Withoutontologies, there really can be no semantics. The ontologies ARE the semantics,they define the meanings that are so essential for connecting information tocontext.

But there are still few widely used or standardized ontologies. Andgetting people to agree on common ontologies is not generally easy. Everyonehas their own way of describing things, their own worldview, and let’s face itnobody wants to use somebody else’s worldview instead of their own.Furthermore, the world is very complex and to adequately describe all the knowledgethat comprises what is thought of as “common sense” would require a very largeontology (and in fact, such an ontology exists – it’s called Cyc and it is solarge and complex that only experts can really use it today).

Even to describe the knowledge ofjust a single vertical domain, such as medicine, is extremely challenging. Tomake matters worse, the tools for authoring ontologies are still very hard touse – one has to understand the OWL language and difficult, buggy ontologyauthoring tools in order to use them. Domain experts who are non-technical andnot trained in formal reasoning or knowledge representation may find theprocess of designing ontologies frustrating using current tools. What is needed are commercial quality tools for buildingontologies that hide the underlying complexity so that people can just pourtheir knowledge into them as easily as they speak. That’s still a ways off, butnot far off. Perhaps ten years at the most.

Of course the difficulty ofdefining ontologies would be irrelevant if the necessary ontologies alreadyexisted. Perhaps experts could define them and then everyone else could justuse them? There are numerous ontologies already in existence, both on thegeneral level as well as about specific verticals. However in my own opinion,having looked at many of them, I still haven’t found one that has the rightbalance of coverage of the necessary concepts most applications need, andaccessibility and ease-of-use by non-experts. That kind of balance is arequirement for any ontology to really go mainstream.

Furthermore, regarding the presentcrop of ontologies, what is still lacking is standardization. Ontologists havenot agreed on which ontologies to use. As a result it’s anybody’s guess whichontology to use when writing a semantic application and thus there is a highdegree of ontology diversity today. Diversity is good, but too much diversityis chaos.

Applications that use differentontologies about the same things don’t automatically interoperate unless theirontologies have been integrated. This is similar to the problem of databaseintegration in the enterprise. In order to interoperate, different applicationsthat use different data schemas for records about the same things, have to bemapped to each other somehow – either at the application-level or the data-level.This mapping can be direct or through some form of middleware.

Ontologies canbe used as a form of semantic middleware, enabling applications to be mapped atthe data-level instead of the applications-level. Ontologies can also be usedto map applications at the applications level, by making ontologies of Webservices and capabilities, by the way. This is an area in which a lot ofresearch is presently taking place.

The OWL language can expressmappings between concepts in different ontologies. But if there are manyontologies, and many of them partially overlap, it is a non-trivial task toactually make the mappings between their concepts.

Even though concept A inontology one and concept B in ontology two may have the same names, and evensome of the same properties, in the context of the rest of the concepts intheir respective ontologies they may imply very different meanings. So simplymapping them as equivalent on the basis of their names is not adequate, theirconnections to all the other concepts in their respective ontologies have to beconsidered as well. It quickly becomes complex. There are some potential waysto automate the construction of mappings between ontologies however – but theyare still experimental. Today, integrating ontologies requires the help ofexpert ontologists, and to be honest, I’m not sure even the experts have itfigured out. It’s more of an art than a science at this point.

Darwinian Selection of Ontologies

All that is needed for mainstream adoption to begin is for a largebody of mainstream content to become semantically tagged andaccessible. This will cause whatever ontology is behind that content to become popular.

When developers see that there is significant content andtraction around aparticular ontology, they will use that ontology for their ownapplicationsabout similar concepts, or at least they will do the work of mappingtheir ownontology to it, and in this way the world will converge in a Darwinianfashionaround a few main ontologies over time.

These main ontologies will then beworth thetime and effort necessary to integrate them on a semantic level,resulting in acohesive Semantic Web. We may in fact see Darwinian natural selection take place not just at the ontology level, but at the level of pieces of ontologies.

A certain ontology may do a good job of defining what a person is, while another may do a good job of defining what a company is. These definitions may be used for a lot of content, and gradually they will become common parts of an emergent meta-ontology comprised of the most-popular pieces from thousands of ontologies. This could be great or it could be a total mess. Nobody knows yet. It’s a subject for further research.

Making Sense of Ontologies

Since ontologies are so important,it is helpful to actually understand what an ontology really is, and what itlooks like. An ontology is a system of formally defined related concepts. Forexample, a simple ontology is this set of statements such as this:

A human is a living thing.

A person is a human.

A person may have a first name.

A person may have a last name.

A person must have one and only onedate of birth.

A person must have a gender.

A person may be socially related toanother person.

A friendship is a kind of socialrelationship.

A romantic relationship is a kindof friendship.

A marriage is a kind of romanticrelationship.

A person may be in a marriage withonly one other person at a time.

A person may be employed by anemployer.

An employer may be a person or anorganization.

An organization is a group ofpeople.

An organization may have a productor a service.

A company is a type organization.

We’ve just built a simple ontologyabout a few concepts: humans, living things, persons, names, socialrelationships, marriages, employment, employers, organizations, groups,products and services. Within this system of concepts there is particular logic,some constraints, and some structure. It may or may not correspond to yourworldview, but it is a worldview that is unambiguously defined, can becommunicated, and is internally logically consistent, and that is what isimportant.

The Semantic Web approach providesan open-standard language, OWL, for defining ontologies. OWL also provides fora way to define instances of ontologies. Instances are assertions within theworldview that a given ontology provides. In other words OWL provides a meansto make statements that connect information to the ontology so that softwarecan understand its meaning unambiguously. For example, below is a set ofstatements based on the above ontology:

There exists a person x.

Person x has a first name “Sue”

Person x  has a last name “Smith”

Person x has a full name “Sue Smith”

Sue Smith was born on June 1, 2005

Sue Smith has a gender: female

Sue Smith has a friend: Jane, who isanother person.

Sue Smith is married to: Bob, anotherperson.

Sue Smith is employed by Acme, Inc, a company

Acme Inc. has a product, Widget2.0.

The set of statements above, plusthe ontology they are connected to, collectively comprise a knowledge basethat, if represented formally in the OWL markup language, could be understoodby any application that speaks OWL in the precise manner that it was intendedto be understood.

Making Metadata

The OWL language provides a way tomarkup any information such as a data record, an email message or a Web pagewith metadata in the form of statements that link particular words or phrasesto concepts in the ontology. When software applications that understand OWLencounter the information they can then reference the ontology and figure outexactly what the information means – or at least what the ontology says that itmeans.

But something has to add thesesemantic metadata statements to the information – and if it doesn’t add them or adds thewrong ones, then software applications that look at the information will getthe wrong idea. And this is another challenge – how will all this metadata getcreated and added into content? People certainly aren’t going to add it all byhand!

Fortunately there are many ways tomake this easier. The best approach is to automate it using special softwarethat goes through information, analyzes the meaning and adds semantic metadataautomatically. This works today, but the software has to be trained or providedwith rules and that takes some time. It also doesn’t scale cost-effectively tovast data-sets.

Alternatively, individuals can beprovided with ways to add semantics themselves as they author information. Whenyou post your resume in a semantically-aware job board, you could fill out aform about each of your past jobs, and the job board would connect that data toappropriate semantic concepts in an underlying employment ontology. As anend-user you would just fill out a form like you are used to doing;under-the-hood the job board would add the semantics for you.

Another approach is to leveragecommunities to get the semantics. We already see communities that are addingbasic metadata “tags” to photos, news articles and maps. Already a few simpletypes of tags are being used pseudo-semantically: subject tags and geographicaltags. These are primitive forms of semantic metadata. Although they are notexpressed in OWL or connected to formal ontologies, they are at leastsemantically typed with prefixes or by being entered into fields or specificnamespaces that define their types.

Tagging by Example

There may also be another solution to the problem of how to add semantics to content in the not to distant future. Once asuitable amount of content has been marked up with semantic metadata,it may be possible, through purely statistical forms of machinelearning, for software to begin to learn how to do a pretty good job ofmarking up new content with semantic metadata.

For example, if thestring “Nova Spivack” is often marked up with semantic metadata statingthat it indicates a person, and not just any person but a specificperson that is abstractly represented in a knowledge base somewhere,then when software applications encounter a new non-semanticallyenhanced document containing strings such as “Nova Spivack” or”Spivack, Nova” they can make a reasonably good guess that thisindicates that same specific person, and they can add the necessarysemantic metadata to that effect automatically.

As more and more semanticmetadata is added to the Web and made accessible it constitutes a statisticaltraining set that can be learned and generalized from. Although humansmay need to jump-start the process with some manually semantic tagging,it might not be long before software could assist them and eventuallydo all the tagging for them. Only in special cases would software needto ask a human for assistance — for example when totally new terms orexpressions were encountered for the first several times.

The technology for doing this learning already exists — and actually it’s not very different from how search engines like Google measure the community sentiment around web pages. Each time something is semantically tagged with a certain meaning that constitutes a “vote” for it having that meaning. The meaning that gets the most votes wins. It’s an elegant, Darwinian, emergent approach to learning how to automatically tag the Web.

One this is certain, if communities were able to tagthings with more types of tags, and these tags were connected to ontologies andknowledge bases, that would result in a lot of semantic metadata being added tocontent in a completely bottom-up, grassroots manner, and this in turn would enable this process to start to become automated or at least machine-augmented.

Getting the Process Started

But making the userexperience of semantic tagging easy (and immediately beneficial) enough that regular people will do it, is a challenge that has yet to be solved.However, it will be solved shortly. It has to be. And many companies andresearchers know this and are working on it right now. This does have to be solved to get the process of jump-starting the Semantic Web started.

I believe that the Tools Problem – the lack of commercial grade tools forbuilding semantic applications – is essentially solved already (although theproducts have not hit the market yet; they will within a few years at most).The Ontology Problem is further from being solved. I think the way this problemwill be solved is through a few “killer apps” that result in the building up ofa large amount of content around particular ontologies within particular onlineservices.

Where might we see this content initially arising? In my opinion it will most likely be within vertical communities of interest, communities of practice, and communities of purpose. Within such communities there is a need to create a common body of knowledge and to make that knowledge more accessible, connected and useful.

The Semantic Web can really improve the quality of knowledge and user-experience within these domains. Because they are communities, not just static content services, these organizations are driven by user-contributed content — users play a key role in building content and tagging it. We already see this process starting to take place in communities such as Flickr, del.icio.us, the Wikipedia and Digg. We know that communities of people do tag content, and consume tagged content, if it is easy and beneficial enough for to them to do so.

In the near future we may see miniature Semantic Webs arising around particular places, topics and subject areas, projects, and other organizations. Or perhaps, like almost every form of new media in recent times, we may see early adoption of the Semantic Web around online porn — what might be called “the sementic web.”

Whether you like it or not, it is a fact that pornography was one of the biggest drivers of early mainstream adoption of personal video technology, CD-ROMs, and also of the Internet and the Web.

But I think it probably is not necessary this time around. While, I’m sure that the so-called “sementic web” could become better from the Semantic Web, it isn’t going to be the primary driver of adoption of the Semantic Web. That’s probably a good thing — the world can just skip over that phase of development and benefit from this technology with both hands so to speak.

The World Wide Database

In some ways one could think of theSemantic Web as “the world wide database” – it does for the meaning of data records what theWeb did for the formatting documents. But that’s just the beginning. It actually turnsdocuments into richer data records. It turns unstructured data into structureddata. All data becomes structured data in fact. The structure is not merelydefined structurally, but it is defined semantically.

In other words, it’s notmerely that for example, a data record or document can be defined in such a wayas to specify that it contains a certain field of data with a certain label ata certain location – it defines what that field of data actually means in anunambiguous, machine understandable way. If all you want is a Web of data,XML is good enough. But if you want to make that data interoperable and machineunderstandable then you need RDF and OWL – the Semantic Web.

Like any database,the Semantic Web, or rather the myriad mini-semantic-webs that will comprise it,have to overcome the challenge of data integration. Ontologies provide a betterway to describe and map data, but the data still has to be described andmapped, and this does take some work. It’s not a magic bullet.

The Semantic Webmakes it easier to integrate data, but it doesn’t completely remove the dataintegration problem altogether. I think the eventual solution to this problemwill combine technology and community folksonomy oriented approaches.

The Semantic Web in HistoricalContext

Let’s transition now and zoom out to see the bigger picture. The Semantic Webprovides technologies for representing and sharing knowledge in new ways. Inparticular, it makes knowledge more accessible to software, and thus to otherpeople. Another way of saying this is that it liberates knowledge fromparticular human minds and organizations – it provides a way to make knowledgeexplicit, in a standardized format that any application can understand. This isquite significant. Let’s put this in historical perspective.

Before the invention of the printing press, there were two ways to spreadknowledge – one was orally, the other was in some symbolic form such as art orwritten manuscripts. The oral transmission of knowledge had limited range and ahigh error-rate, and the only way to learn something was to meet someone whoknew it and get them to tell you. The other option, symbolic communicationthrough art and writing, provided a means to communicate knowledgeindependently of particular people – but it was only feasible to produce a fewcopies of any given artwork or manuscript because they had to be copied byhand. So the transmission of knowledge was limited to small groups or at leastsmall audiences. Basically, the only way to get access to this knowledge was tobe one of the lucky few who could acquire one of its rare physical copies.

The invention of the printing press changed this – for the first timeknowledge could be rapidly and cost-effectively mass-produced and mass-distributed.Printing made it possible to share knowledge with ever-larger audiences. Thisenabled a huge transformation for human knowledge, society, government,technology – really every area of human life was transformed by thisinnovation.

The World Wide Web made the replication and distribution of knowledge eveneasier – With the Web you don’t even have to physically print or distributeknowledge anymore, the cost of distribution is effectively zero, and everyonehas instant access to everything from anywhere, anytime. That’s a lot betterthan having to lug around a stack of physical books. Everyone potentially haswhatever knowledge they need with no physical barriers. This has been anotherhuge transformation for humanity – and it has affected every area of humanlife. Like the printing press, the Web fundamentally changed the economics ofknowledge.

The Semantic Web is the next big step in this process – it will make all theknowledge of the human race accessible to software. For the first time,non-human things (software applications) will be able to start working withhuman knowledge to do things (for humans) on their own. This is a big leap – aleap like the emergence of a new species, or the symbiosis of two existingspecies into a new form of life.

The printing press and the Web changed the economics of replicating,distributing and accessing knowledge. The Semantic Web changes the economics ofprocessing knowledge. Unlike the printing press and the Web, the Semantic Webenables knowledge to be processed by non-human things.

In other words, humans don’t have to do all the thinking on their own, theycan be assisted by software. Of course we humans have to at least first createthe software (until we someday learn to create software that is smart enough tocreate software too), and we have to create the ontologies necessary for thesoftware to actually understand anything (until we learn to create software thatis smart enough to create ontologies too), and we have to add the semanticmetadata to our content in various ways (until our software is smart enough todo this for us, which it almost is already). But once we do the initial work ofmaking the ontologies and software, and adding semantic metadata, the systemstarts to pick up speed on its own, and over time the amount of work we humanshave to do to make it all function decreases. Eventually, once the system hasencoded enough knowledge and intelligence, it starts to function withoutneeding much help, and when it does need our help, it will simply ask us andlearn from our answers.

This may sound like science-fiction today, but in fact it a lot of this isalready built and working in the lab. The big hurdle is figuring out how to getthis technology to mass-market. That is probably as hard as inventing thetechnology in the first place. But I’m confident that someone will solve iteventually.

Once this happens the economics of processing knowledge will truly bedifferent than it is today. Instead of needing an actual real-live expert, theknowledge of that expert will be accessible to software that can act as theirproxy – and anyone will be able to access this virtual expert, anywhere,anytime. It will be like the Web – but instead of just information beingaccessible, the combined knowledge and expertise of all of humanity will alsobe accessible, and not just to people but also to software applications.

The Question of Consciousness

The Semantic Web literally enables humans to share their knowledge with eachother and with machines. It enables the virtualization of human knowledge andintelligence. With respect to machines, in doing this, it will lend machines“minds” in a certain sense – namely in that they will at least be able tocorrectly interpret the meaning of information and replicate the expertise ofexperts.

But will these machine-minds be conscious? Will they be aware of themeanings they interpret, or will they just be automatons that are simplyfollowing instructions without any awareness of the meanings they areprocessing? I doubt that software will ever be conscious, because from what Ican tell consciousness — or what might be called the sentient awareness ofawareness itself as well as other things that are sensed — is an immaterialphenomena that is as fundamental as space, time and energy — or perhaps evenmore fundamental. But this is just my personal opinion after having searchedfor consciousness through every means possible for decades. It just cannot befound to be something, yet it is definitely and undeniably taking place.

Consciousness can be exemplified through the analogy of space (but unlikespace, consciousness has this property of being aware, it’s not a mere lifelessvoid). We all agree space is there, but nobody can actually point to itsomewhere, and nobody can synthesize space. Space is immaterial andfundamental. It is primordial. So is electricity. Nobody really knows whatelectricity is ultimately, but if you build the right kind of circuit you canchannel it and we’ve learned a lot about how to do that.

Perhaps we may figure out how to channel consciousness like we channelelectricity with some sort of synthetic device someday, but I think that ishighly unlikely. I think if you really want to create consciousness it’s mucheasier and more effective to just have children. That’s something ordinarymortals can do today with the technology they were born with. Of course whenyou have children you don’t really “create” their consciousness, it seems to bethere on its own. We don’t really know what it is or where it comes from, orwhen it arises there. We know very little about consciousness today.Considering that it is the most fundamental human experience of all, it isactually surprising how little we know about it!

In any case, until we truly delve far more deeply into the nature of themind, consciousness will be barely understood or recognized, let aloneexplained or synthesized by anyone. In many eastern civilizations there aremulti-thousand year traditions that focus quite precisely on the nature ofconsciousness. The major religions have all universally concluded thatconsciousness is beyond the reach of science, beyond the reach of concepts,beyond the mind entirely. All those smart people analyzing consciousness for solong, and with such precision, and so many methods of inquiry, may have a pointworth listening to.

Whether or not machines will ever actually “know” or be capable of beingconscious of that meaning or expertise is a big debate, but at least we can allagree that they will be able to interpret the meaning of information and rulesif given the right instructions. Without having to be conscious, software willbe able to process semantics quite well — this has already been proven. It’sworking today.

While consciousness is and may always be a mystery that we cannot synthesize– the ability for software to follow instructions is an established fact. Inits most reduced form, the Semantic Web just makes it possible to providericher kinds of instructions. There’s no magic to it. Just a lot of details. Infact, to play on a famous line, “it’s semantics all the way down.”

The Semantic Web does not require that we make conscious software. It justprovides a way to make slightly more intelligent software. There’s a bigdifference. Intelligence is simply a form of information processing, for themost part. It does not require consciousness — the actual awareness of what isgoing on — which is something else altogether.

While highly intelligentsoftware may need to sense its environment and its own internal state andreason about these, it does not actually have to be conscious to do this. Theseoperations are for the most part simple procedures applied vast numbers of timeand in complex patterns. Nowhere in them is there any consciousness nor doesconsciousness suddenly emerge when suitable levels of complexity are reached.

Consciousness is something quite special and mysterious. And fortunately forhumans, it is not necessary for the creation of more intelligent software, noris it a byproduct of the creation of more intelligent software, in my opinion.

The Intelligence of the Web

So the real point of the Semantic Web is that it enables the Web to becomemore intelligent. At first this may seem like a rather outlandish statement,but in fact the Web is already becoming intelligent, even without the SemanticWeb.

Although the intelligence of the Web is not very evident at first glance,nonetheless it can be found if you look for it. This intelligence doesn’t existacross the entire Web yet, it only exists in islands that are few and farbetween compared to the vast amount of information on the Web as a whole. Butthese islands are growing, and more are appearing every year, and they arestarting to connect together. And as this happens the collective intelligenceof the Web is increasing.

Perhaps the premier example of an “island of intelligence” is theWikipedia, but there are many others: The Open Directory, portals such as Yahooand Google, vertical content providers such as CNET and WebMD, commercecommunities such as Craigslist and Amazon, content oriented communities such asLiveJournal, Slashdot, Flickr and Digg and of course the millions of discussionboards scattered around the Web, and social communities such as MySpace andFacebook. There are also large numbers of private islands of intelligence onthe Web within enterprises — for example the many online knowledge andcollaboration portals that exist within businesses, non-profits, andgovernments.

What makes these islands “intelligent” is that they are places where people(and sometimes applications as well) are able to interact with each other tohelp grow and evolve collections of knowledge. When you look at them close-upthey appear to be just like any other Web site, but when you look at what theyare doing as a whole – these services are thinking.They are learning, self-organizing, sensing their environments, interpreting,reasoning, understanding, introspecting, and building knowledge. These are theactivities of minds, of intelligent systems.

The intelligence of a system such as the Wikipedia exists on several levels– the individuals who author and edit it are intelligent, the groups that helpto manage it are intelligent, and the community as a whole – which isconstantly growing, changing, and learning – is intelligent.

Flickr and Digg also exhibit intelligence. Flickr’s growing system of tagsis the beginnings of something resembling a collective visual sense organ onthe Web. Images are perceived, stored, interpreted, and connected to conceptsand other images. This is what the human visual system does. Similarly, Digg isa community that collectively detects, focuses attention on, and interpretscurrent news. It’s not unlike a primitive collective analogue to the humanfacility for situational awareness.

There are many other examples of collective intelligence emerging on theWeb. The Semantic Web will add one more form of intelligent actor to the mix –intelligent applications. In the future, after the Wikipedia is connected tothe Semantic Web, as well as humans, it will be authored and edited by smartapplications that constantly look for new information, new connections, and newinferences to add to it.

Although the knowledge on the Web today is still mostly organized withindifferent islands of intelligence, these islands are starting to reach out andconnect together. They are forming trade-routes, connecting their economies,and learning each other’s languages and cultures. The next-step will be forthese islands of knowledge to begin to share not just content and services, butalso their knowledge — what they know about their content and services. The SemanticWeb will make this possible, by providing an open format for the representationand exchange of knowledge and expertise.

When applications integrate their content using the Semantic Web they willalso be able to integrate their context, their knowledge – this will make thecontent much more useful and the integration much deeper. For example, when anapplication imports photos from another application it will also be able toimport semantic metadata about the meaning and connections of those photos.Everything that the community and application know about the photos in theservice that provides the content (the photos) can be shared with the servicethat receives the content. Better yet, there will be no need for customapplication integration in order for this to happen: as long as both servicesconform to the open standards of the Semantic Web the knowledge is instantlyportable and reusable.

Freeing Intelligence from Silos

Today much of the real value of the Web (and in the world) is still lockedaway in the minds of individuals, the cultures of groups and organizations, andapplication-specific data-silos. The emerging Semantic Web will begin to unlockthe intelligence in these silos by making the knowledge and expertise theyrepresent more accessible and understandable.

It will free knowledge and expertise from the narrow confines of individualminds, groups and organizations, and applications, and make them not only moreinteroperable, but more portable. It will be possible for example for a personor an application to share everything they know about a subject of interest aseasily as we share documents today. In essence the Semantic Web provides acommon language (or at least a common set of languages) for sharing knowledgeand intelligence as easily as we share content today.

The Semantic Web also provides standards for searching and reasoning moreintelligently. The SPARQL query language enables any application to ask forknowledge from any other application that speaks SPARQL. Instead of merekeyword search, this enables semantic search. Applications can search forspecific types of things that have particular attributes and relationships toother things.

In addition, standards such as SWRL provide formalisms for representing andsharing axioms, or rules, as well. Rules are a particular kind of knowledge –and there is a lot of it to represent and share, for example proceduralknowledge, and logical structures about the world. An ontology provides a meansto describe the basic entities, their attributes and relations, but rulesenable you to also make logical assertions and inferences about them. Withoutgoing into a lot of detail about rules and how they work here, the importantpoint to realize is that they are also included in the framework. All forms ofknowledge can be represented by the Semantic Web.

Zooming Way, Waaaay Out

So far in this article, I’ve spenta lot of time talking about plumbing – the pipes, fluids, valves, fixtures,specifications and tools of the Semantic Web. I’ve also spent some time onillustrations of how it might be useful in the very near future to individuals,groups and organizations. But where is it heading after this? What is thelong-term potential of this and what might it mean for the human race on ahistorical time-scale?

For those of you who would prefer not to speculate, stop reading here. Forthe rest of you, I believe that the true significance of the Semantic Web, on along-term timescale is that it provides an infrastructure that will enable theevolution of increasingly sophisticated forms of collective intelligence. Ultimatelythis will result in the Web itself becoming more and more intelligent, untilone day the entire human species together with all of its software andknowledge will function as something like a single worldwide distributed mind –a global mind.

Just the like the mind of a single human individual, the global mind will bevery chaotic, yet out of that chaos will emerge cohesive patterns of thoughtand decision. Just like in an individual human mind, there will be feedbackbetween different levels of order – from individuals to groups to systems ofgroups and back down from systems of groups to groups to individuals. Becauseof these feedback loops the system will adapt to its environment, and to itsown internal state.

The coming global mind will collectively exhibit forms of cognition andbehavior that are the signs of higher-forms of intelligence. It will form andreact to concepts about its “self” – just like an individual human mind. Itwill learn and introspect and explore the universe. The thoughts it thinks maysometimes be too big for any one person to understand or even recognize them –they will be comprised of shifting patterns of millions of pieces of knowledge.

The Role of Humanity

Every person on the Internet will be a part of the global mind. Andcollectively they will function as its consciousness. I do not believe some newform of consciousness will suddenly emerge when the Web passes some thresholdof complexity. I believe that humanity IS the consciousness of the Web anduntil and unless we ever find a way to connect other lifeforms to the Web, orwe build conscious machines, humans will be the only form of consciousness ofthe Web.

When I say that humans will function as the consciousness of the Web I meanthat we will be the things in the system that know. The knowledge of theSemantic Web is what is known, but what knows that knowledge has to besomething other than knowledge. A thought is knowledge, but what knows thatthought is not knowledge, it is consciousness, whatever that is. We can figureout how to enable machines to represent and use knowledge, but we don’t knowhow to make them conscious, and we don’t have to. Because we are alreadyconscious.

As we’ve discussed earlier in this article, we don’t need conscious machines, we just need more intelligent machines.Intelligence – at least basic forms of it – does not require consciousness. It may be the case that the very highest forms of intelligence require or are capable of consciousness. This may mean that software will never achieve the highest levels of intelligence and probably guaranteesthat humans (and other conscious things) will always play a special role in theworld; a role that no computer system will be able to compete with. We providethe consciousness to the system. There may be all sorts of other intelligent,non-conscious software applications and communities on the Web; in fact therealready are, with varying degrees of intelligence. But individual humans, andgroups of humans, will be the only consciousness on the Web.

The Collective Self

Although the software of the Semantic Web will not be conscious we can say that system as a whole contains or is conscious to the extent that human consciousnesses are part of it. And like most conscious entities, it may also start to be self-conscious.

If the Web ever becomes a global mind as I am predicting, will it have a“self?” Will there be a part of the Web that functions as its central self-representation?Perhaps someone will build something like that someday, or perhaps it will evolve.Perhaps it will function by collecting reports from applications and people inreal-time – a giant collective zeitgeist.

In the early days of the Web portals such as Yahoo! provided this function — they were almost real-time maps of the Web and what was happening. Today making such a map is nearly impossible, but services such as Google Zeitgeist at least attempt to provide approximations of it. Perhaps through random sampling it can be done on a broader scale.

My guess is that the global mind will need a self-representation at somepoint. All forms of higher intelligence seem to have one. It’s necessary forunderstanding, learning and planning. It may evolve at first as a bunch ofcompeting self-representations within particular services or subsystems withinthe collective. Eventually they will converge or at least narrow down to just afew major perspectives. There may also be millions of minor perspectives thatcan be drilled down into for particular viewpoints from these top-level “portals.”

The collective self, will function much like the individual self – as amirror of sorts. Its function is simply to reflect. As soon as it exists theentire system will make a shift to a greater form of intelligence – because forthe first time it will be able to see itself, to measure itself, as a whole. Itis at this phase transition when the first truly global collective self-mirroring function evolves, that we can say that the transition from a bunch of cooperating intelligent parts toa new intelligent whole in its own right has taken place.

I think that the collective self, even if it converges on a few majorperspectives that group and summarize millions of minor perspectives, will becommunity-driven and highly decentralized. At least I hope so – because theself-concept is the most important part of any mind and it should be designedin a way that protects it from being manipulated for nefarious ends. At least Ihope that is how it is designed.

Programming the Global Mind

On the other hand, there are times when a little bit of adjustment or guidance iswarranted – just as in the case of an individual mind, the collective selfdoesn’t merely reflect, it effectively guides the interpretation of the pastand present, and planning for the future.

One way to change the direction ofthe collective mind, is to change what is appearing in the mirror of thecollective self. This is a form of programming on a vast scale – When thisprogramming is dishonest or used for negative purposes it is called “propaganda,” but there are cases whereit can be done for beneficial purposes as well. An example of this today ispublic service advertising and educational public television programming. Allforms of mass-media today are in fact collective social programming. When yourealize this it is not surprising that our present culture is violent andmessed up – just look at our mass-media!

In terms of the global mind, ideally one would hope that it would be able tolearn and improve over time. One would hope that it would not have the collective equivalent of psycho-social disorders. To facilitate this, just like any form of higherintelligence, it may need to be taught, and even parented a bit. It also mayneed a form of therapy now and then. These functions could be provided by thepeople who participate in it. Again, I believe that humans serve a vital and irreplaceablerole in this process.

How It All Might Unfold

Now how is this all going to unfold? I believe that there are a number ofkey evolutionary steps that Semantic Web will go through as the Web evolvestowards a true global mind:

1. Representing individual knowledge. The first step is to make individuals’knowledge accessible to themselves. As individuals become inundated withincreasing amounts of information, they will need better ways of managing it,keeping track of it, and re-using it. They will (or already do) need”personal knowledge management.”

2. Connecting individual knowledge. Next, once individual knowledge isrepresented, it becomes possible to start connecting it and sharing it acrossindividuals. This stage could be called “interpersonal knowledgemanagement.”

3. Representing group knowledge. Groups of individuals also need ways ofcollectively representing their knowledge, making sense of it, and growing itover time. Wikis and community portals are just the beginning. The Semantic Webwill take these “group minds” to the next level — it will make the collective knowledge ofgroups far richer and more re-usable.

4. Connecting group knowledge. This step is analogous to connectingindividual knowledge. Here, groups become able to connect their knowledge togetherto form larger collectives, and it becomes possible to more easily access andshare knowledge between different groups in very different areas of interest.

5. Representing the knowledge of the entire Web. This stage — what might becalled “the global mind” — is still in the distant future, but atthis point in the future we will begin to be able to view, search, and navigatethe knowledge of the entire Web as a whole. The distinction here is thatinstead of a collection of interoperating but separate intelligentapplications, individuals and groups, the entire Web itself will begin tofunction as one cohesive intelligent system. The crucial step that enables thisto happen is the formation of a collective self-representation. This enablesthe system to see itself as a whole for the first time.

How it May be Organized

I believe the global mind will be organized mainly in the form of bottom-up and lateral, distributed emergent computation andcommunity — but it will be facilitated by certain key top-down services thathelp to organize and make sense of it as a whole. I think this future Web willbe highly distributed, but will have certain large services within it as well– much like the human brain itself, which is organized into functionalsub-systems for processes like vision, hearing, language, planning, memory,learning, etc.

As the Web gets more complex there will come a day when nobody understandsit anymore – after that point we will probably learn more about how the Web isorganized by learning about the human mind and brain – they will be quitesimilar in my opinion. Likewise we will probably learn a tremendous amountabout the functioning of the human brain and mind by observing how the Webfunctions, grows and evolves over time, because they really are quite similarin at least an abstract sense.

The internet and its software and content is like a brain, and the state ofits software and the content is like its mind. The people on the Internet arelike its consciousness. Although these are just analogies, they are actuallyuseful, at least in helping us to envision and understand this complex system. Asthe field of general systems theory has shown us in the past, systems at verydifferent levels of scale tend to share the same basic characteristics and obeythe same basic laws of behavior. Not only that, but evolution tends to convergeon similar solutions for similar problems. So these analogies may be more thanjust rough approximations, they may be quite accurate in fact.

The future global brain will require tremendous computing and storageresources — far beyond even what Google provides today. Fortunately as Moore’s Law advances thecost of computing and storage will eventually be low enough to do thiscost-effectively. However even with much cheaper and more powerful computingresources it will still have to be a distributed system. I doubt that therewill be any central node because quite simply no central solution will be ableto keep up with all the distributed change taking place. Highly distributed problemsrequire distributed solutions and that is probably what will eventually emergeon the future Web.

Someday perhaps it will be more like a peer-to-peer network, comprised ofapplications and people who function sort of like the neurons in the human brain.Perhaps they will be connected and organized by higher-level super-peers orsuper-nodes which bring things together, make sense of what is going on andcoordinate mass collective activities. But even these higher-level serviceswill probably have to be highly distributed as well. It really will bedifficult to draw boundaries between parts of this system, they will all beconnected as an integral whole.

In fact it may look very much like a grid computing architecture – in whichall the services are dynamically distributed across all the nodes such that atany one time any node might be working on a variety of tasks for differentservices. My guess is that because this is the simplest, most fault-tolerant,and most efficient way to do mass computation, it is probably what will evolvehere on Earth.

The Ecology of Mind

Where we are today in this evolutionary process is perhaps equivalent to therise of early forms of hominids. Perhaps Austrolapithecus or Cro-Magnon, ormaybe the first Homo Sapiens. Compared to early man, the global mind is like the rise of 21stcentury mega-cities. A lot of evolution has to happen to get there. But itprobably will happen, unless humanity self-destructs first,which I sincerely hope we somehow manage to avoid. And this brings me to afinal point. This vision of the future global mind is highly technological;however I don’t think we’ll ever accomplish it without a new focus on ecology.

Ecology probably conjures up images of hippies and biologists, or maybehippies who are biologists, or at least organic farmers, for most people, but infact it is really the science of living systems and how they work. And anysystem that includes living things is a living system. This means that the Webis a living system and the global mind will be a living system too. As a living system, the Web is an ecosystem and is alsoconnected to other ecosystems. In short, ecology is absolutely essential tomaking sense of the Web, let alone helping to grow and evolve it.

In many ways the Semantic Web and the collective minds, and the global mind,that it enables, can be seen as an ecosystem of people, applications,information and knowledge. This ecosystem is very complex, much like naturalecosystems in the physical world. An ecosystem isn’t built, it’s grown, andevolved. And similarly the Semantic Web, and the coming global mind, will notreally be built, they will be grown and evolved. The people and organizationsthat end up playing a leading role in this process will be the ones thatunderstand and adapt to the ecology most effectively.

In my opinion ecology is going to be the most important science anddiscipline of the 21st century – it is the science of healthysystems. What nature teaches us about complex systems can be applied to everykind of system – and especially the systems we are evolving on the Web. Inorder to ever have a hope of evolving a global mind, and all the wonderfullevels of species-level collective intelligence that it will enable, we have tonot destroy the planet before we get there. Ecology is the science that cansave us, not the Semantic Web (although perhaps by improving collectiveintelligence, it can help).

Ecology is essentially the science of community – whether biological,technological or social. And community is a key part of the Semantic Web atevery level: communities of software, communities of people, and communities ofgroups. In the end the global mind is the ultimate human community. It is thereward we get for finally learning how to live together in peace and balancewith our environment.

The Necessity of Sustainability

The point of this discussion of the relevance of ecology to the future ofthe Web, and my vision for the global mind, is that I think that it is clearthat if the global mind ever emerges it will not be in a world that is anythinglike what we might imagine. It won’t be like the Borg in Star Trek, it won’t belike living inside of a machine. Humans won’t be relegated to the roles ofslaves or drones. Robots won’t be doing all the work. The entire world won’t becoated with silicon. We won’t all live in a virtual reality. It won’t be one ofthese technological dystopias.

In fact, I think the global mind can only come to pass in a much greener,more organic, healthier, more balanced and sustainable world. Because it willtake a long time for the global mind to emerge, if humanity doesn’t figure outhow to create that sort of a world, it will wipe itself out sooner or later,but certainly long before the global mind really happens. Not only that, butthe global mind will be smart by definition, and hopefully this intelligencewill extend to helping humanity manage its resources, civilizations andrelationships to the natural environment.

The Smart Environment

The global mind also needs a global body so to speak. It’s not going to bean isolated homunculus floating in a vat of liquid that replaces the physicalworld! It will be a smart environment that ubiquitously integrates with ourphysical world. We won’t have to sit in front of computers or deliberatelylogon to the network to interact with the global mind. It will be everywhere.

The global mind will be physically integrated into furniture, houses,vehicles, devices, artworks, and even the natural environment. It will sensethe state of the world and different ecosystems in real-time and alert humansand applications to emerging threats. It will also be able to allocateresources intelligently to compensate for natural disasters, storms, andenvironmental damage – much in the way that the air traffic control systemsallocates and manages airplane traffic. It won’t do it all on its own, humansand organizations will be a key part of the process.

Someday the global mind may even be physically integrated into our bodiesand brains, even down the level of our DNA. It may in fact learn how to curediseases and improve the design of the human body, extending our lives, sensorycapabilities, and cognitive abilities. We may be able to interact with it bythought alone. At that point it will become indistinguishable from a limitedfrom of omniscience, and everyone may have access to it. Although it will onlyextend to wherever humanity has a presence in the universe, within thatboundary it will know everything there is to know, and everyone will be able toknow any of it they are interested in.

Enabling a Better World

By enabling greater forms of collective intelligence to emerge we really arehelping to make a better world, a world that learns and hopefully understandsitself well enough to find a way to survive. We’re building something thatsomeday will be wonderful – far greater than any of us can imagine. We’re helpingto make the species and the whole planet more intelligent. We’re building thetools for the future of human community. And that future community, if it ever arrives,will be better, more self-aware, more sustainable than the one we live intoday.

I should also mention that knowledge is power, and power can be used forgood or evil. The Semantic Web makes knowledge more accessible. This puts more power in the hands of the many, not just the few. As long as we stick to this vision — we stick to making knowledge open and accessible, using open standards, in as distributed a fashion as we can devise, then the potential power of the Semantic Web will be protected against being coopted or controlled by the few at the expense of the many. This is where technologists really have to be socially responsible when making development decisions. It’s important that we build a more open world, not a less open world. It’s important that we build a world where knowledge, integration and unification are balanced with respect for privacy, individuality, diversity and freedom of opinion.

But I am not particularly worried that the Semantic Web and the future globalmind will be the ultimate evil – I don’t think it is likely that we will end upwith a system of total control dominated by evil masterminds with powerfulSemantic Web computer systems to do their dirty work. Statistically speaking, criminal empires don’t last very long because theyare run by criminals who tend to be very short-sighted and who also surroundthemselves with other criminals who eventually unseat them, or theyself-destruct. It’s possible that the Semantic Web, like any other technology,may be used by the bad guys to spy on citizens, manipulate the world, and doevil things. But only in the short-term.

In the long-term either our civilization will get tired of endlesssuccessions of criminal empires and realize that the only way to actuallysurvive as a species is to invent a form of government that is immune to beingtaken over by evil people and organizations, or it will self-destruct. Eitherway, that is a hurdle we have to cross before the global mind that I envisioncan ever come about. Many civilizations came before ours, and it is likely thatours will not be the last one on this planet. It may in fact be the case that adifferent form of civilization is necessary for the global mind to emerge, andis the natural byproduct of the emergence of the global mind.

We know that the global mind cannot emerge anytime soon, and therefore, ifit ever emerges then by definition it must be in the context of a civilizationthat has learned to become sustainable. A long-term sustainable civilization is a non-evil civilization. And that is why I think it is a safebet to be so optimistic about the long-term future of this trend.

The Sell-Off of America

This article by Germany’s best-known economics writer provides a fast and high-level overview of how the American empire is losing (has lost?) its economic power. While the dollar is still the world’s currency of choice, the USA no longer controls it. Furthermore, with increasing trade deficits, the outsourcing of labor, and spiraling debt, the US economy is poised on the edge of collapse. And the American middle-class has and will bear the brunt of this shift as it plays out over the next few decades.

Interesting Proposal for Stopping Online Pedophiles

I came across this interesting project that aims to stop online pedophiles by setting 100,000 online honeytraps. It calls for 100,000 volunteer adults to post online profiles in various services posing as minors. They then use these a bait and wait to be contacted by pedophiles. Once contacted they basically set the hook and then do a sting operation with the help of law enforcement agents. The idea is that by doing this the odds of an online profile of a minor actually being a trap, and thus the odds of a pedophile getting caught are increased enough to scare and/or catch the majority of online sexual predators. It’s kind of like a "neighborhood watch" for the Web. I don’t know how legal it is, but at least it makes sense statistically.

Of course, there’s an easier way to catch pedophiles these days: Just visit the US House of Representatives.

A Proposal to Make the Media (and Society) Better

I am concerned by what I’m viewing in our national media lately. Viewed from
outside (and also from wihtin the USA), it would appear that our nation
is obsessed with, and plagued by, an increasing spree of horrible crimes and
abuses of human rights. Is this really what it is like to live in
America, or is this simply the media’s extremely biased reporting?

The media of any social organization, whether a small community or an entire nation, serves as the primary means by which the members of that organization understand and assess the state of the whole. As such the media serves a function to a collective that is similar to "self-awareness" in an individual person. Self-awareness is a feedback loop. It not only serves as a mirror by which an organization can view reflections of its state, it also conditions the planning and execution of further actions by its organization. When people look in the mirror of the media, they not only understand the system they are in, but they also may make choices and further actions based on what they see there.

The pivotal role of media as the central influencer of societal psychology and behavior is often underestimated. And it is through this lens that I view the recent headlines, which are primarily focused on school killing sprees and abuse of children as of late — with sadness and concern. By sensationalizing such tragedies, and providing lurid details over and over again, the media is actually contributing to the ongoing proliferation of these same behaviors. While I’m not suggesting that censorship is a better alternative, I do think that leading mass media organizations, in their blind quest for more attention and ad dollars, are behaving irresponsibly by sensationalizing — indeed almost celebrating — murder, torture, sexual abuse, and other horrible acts as if they are the latest attractions in some kind of societal variety show.

These atrocities should be reported, but the style and manner in which they are reported should be the opposite of sensationalism. They should be more like funerals — which is after all what these stories are really about. There should be an element of sadness, respect for those affected, concern for those who might read or view such images and stories, and restraint. In particular details of how various horrible crimes were planned and executed are not necessary and not beneficial in telling the story — they only serve to inspire, trigger and teach would-be copycats to do the same thing. Furthermore by rewarding perpetrators of such crimes with massive publicity (which in many cases is what such egomaniacs are hoping to achieve by committing their crimes) the media is actually playing right into their hands.

As usual I have a radical proposal to fix this: What if the media gave exposure to stories in direct proportion to the number of people actually affected by those stories?

What if stories were evaluated on several dimensions to determine how much exposure to give them — for example, geographic range, number of people directly impacted, political relevance, etc? The size of a story would therefore be determined by its real effect on the population. Contrast this to what the news media does today where in fact the prominence of a story is often inversely proportional to the number of people directly impacted by it. The media currently amplifies the most shocking news, regardless of how many people are directly affected or involved. A shocking crime that directly affected only 2 people is quickly amplified to the main national headline of the day for a population of 300 million. That is simply out of balance with reality.

For example, horrible as they are, the recent school killings only directly affect a relatively tiny percentage of the national population: So why should they be the main focus of the national news headlines for weeks on end? According to this proposal, such stories would be mere blips on the collective radar. They would not be headline news for the entire nation, and/or if they were they would quickly fall out of the headlines. I’m not downplaying the importance of these stories to the affected people and communities — nor am I downplaying the broader significance of the trends they may be indicative of — I’m simply pointing out that these stories, tragic as they may be, get disproportionately too much national media exposure compared to other stories that affect much larger segments of the population such as for example — stories related to the economy, the environment, the war and upcoming elections, etc.

Continue reading

Diebold Voting Machines Conclusively Hackable

Allegations of electronic tampering with Diebold voting machines in previous elections gained new evidence this week. A research paper by a team of researchers at Princeton University has conclusively demonstrated that Deibold voting machines are trivially easy to hack. The team not only hacked the machines and changed votes, they created a virus that can do it — and that spreads from machine to machine.

Privacy and Search

The recent negative hype about the lack of privacy in search results got me thinking about the needs of online services versus those of individuals. Is there a way to satisfy both constraints?

AOL’s accidental data release was one thing that worried me. Google’s "personal search" feature, where the log of all your searches is displayed, was another. The fact that everything you search for and click on, during your entire life, could potentially be logged, owned, accessed, and shared, by and with parties other than yourself, without your consent or even your knowledge, is a step towards a world I wouldn’t want to live in.

The arguments in favor of allowing this to continue either hinge on commercial needs or homeland security and law enforcement. Regarding commercial needs: just as in other situations where commercial needs pose risks to individual privacy (such as medical records for example), the government needs to step in and regulate if the industry can’t do an adequate job of self-regulating. And regarding industry self-regulation, it can easily become a case of wolves guarding sheep, and so has to be carefully regulated by government on a meta-level. As for the needs of homeland security and law enforcement, access should be strictly regulated (and in theory, it already is). 

The thing is, even if governments and industry stepped up and took responsibility for regulating this situation, one can never be
sure that future regime change, accidents, or individuals or groups
with both access and a motive won’t lead to future privacy violations. As a result
even ironclad assurances, laws, and strict procedures by organizations
and governments, won’t protect anyone against such unknowns. The only truly safe solution is one that puts all of the control, and all of the responsibility and liability, for one’s own private data, in one’s own hands. In a digital world, where everything is potentially recorded and logged forever, this is really important.

The solution is, I think, that individuals, rather than search companies, should own and control their searchstreams and their clickstreams such that they can make use of that information for their own personalization needs, and they can selectively (and either authentically or anonymously) share it with other services if and when they want to. Someone should build an infrastructure that enables this and then make it an API that all services and apps can use. The folks at Attention Trust and Root Markets are on the right track. This is a very interesting business opportunity.

I would like to see a search engine and a search toolboar for Firefox that enable you to search anonymously. I did a little research (on Google, how ironic) and found Proxify, Kaxy and Mezzy. They seem interesting although perhaps a little clunky seeming. What we need is a high-profile, really polished, professional, well-funded, simple anonymous proxy for Google. And a Firefox toolbar to go with it.

If a service like what I am describing existed (and there was some level of independent audit that could assure me that it really didn’t capture or save anything private without my permission — for example if all the code was open source and vetted), then I would definitely always use it instead of going directly to Google. Does it exist already? Let me know. If not, someone should build it. In fact, I wouldn’t mind if it showed me ads, just like Google does. So it could make money from my searching. I would bring my business there as would most people who have educated themselves about this issue.

Finally, wearing my corporate hat for the moment, as someeone building an online service in the search space, if there was a suitable (and that is the key term here…) way that the service my company is building could give individuals control of their private data while also still being able to learn from it in aggregate and/or anonymously for individuals, that would be great. As an online service provider I don’t really want to have to worry about keeping such private information and all the overhead and potential liability that goes with it.

Online services do need to learn from the behavior of their users in order to personalize content and target ads, etc. But they don’t need to necessarily house that data themselves, nor do they need to necessarily be able to key it to the real identities of their users. If there was an infrastructure that enabled my service to learn, personalize and target, without having to hold and manage the dataset underlying that capability, that would actually be a potential savings to my business, and a reduction of risk, and a benefit to my users. The thing is, while early attempts to enable this do exist, they aren’t mature enough to rely on, and nobody knows how well they will scale or whether they will have enough funding and traction to last. So in the meantime those of us building online services are in a gray area — we need certain features for our services to function well, and we also would like to find a way to protect privacy for the individual. This is the connundrum of the moment. It’s a business opportunity for someone out there.

Polling the Global Mind

Grupthink is a service where anyone can create and participate in polls on various subjects. It’s similar to an idea I once had about polling the global mind in real-time (although their system does not show votes happening in real-time, presently). It reminds me of several other Web 2.0 sites, but it’s nicely done. Worth a look.

China About to "Strike Hard" Against Civil Unrest

Chinese officials are preparing to "strike hard" against civil unrest. Are they planning another Tiannamen-style massacre? There is something fundamentally wrong with a government that has to torture, repress, and murder it’s own people (and its neighboring peoples) on a regular, systematic basis in order to maintain stability. China isn’t the only country guilty of such crimes, but they are certainly the most visible. As one of the largest, and most important, nations on earth — and as a key member of the global community — China really should hold itself to a different standard. Just because protestors are comprised of disenfranchised peasants, workers, students and ethnic minorities does not mean that their lives and causes are without value. In fact, the whole point of the Maoist revolution was ostensibly to liberate the lower classes from oppression — so I find it very ironic that the regime that grew out of that revolution is now actively oppressing the very same people it was formed to liberate. Worse still is that the Chinese government — with or without Google’s support — cannot possibly succeed at a policy of repression in the long-term. Eventually, the more acts of violence that are perpetrated against the Chinese people, the more disillusioned those people will become. And eventually the regime will fall out of favor and be dissolved. Since that is the inevitable outcome of the current direction of Chinese policy, why not just spare the needless loss of life, and take a different approach now? The Chinese government should find a way to listen to its people and work with them to build partnerships and solutions from the bottom-up instead of forcing them to comply from the top-down. Anyway, what I say doesn’t matter — I’m sure the Chinese government is not reading this blog. Fortunately, capitalism is like a virus, or an acid, that is already colonizing China and will inevitably result in a shift to something like democracy there, eventually. It’s just too bad that so many people have to suffer in the process.

Big Thinkers' Most Dangerous Ideas

The Edge has published mini-essays by 119 "big thinkers" on their "most dangerous ideas" — fun reading.

The history of science is replete with discoveries
that were considered socially, morally, or emotionally
dangerous in their time; the Copernican and
Darwinian revolutions are the most obvious.
What is your dangerous idea? An idea you think
about (not necessarily one you originated)
that is dangerous not because it is assumed to be false, but because it might be true?

 

Another Actor Running for Office?

Oy Vey. CNN just reported that Virginia Democrats are rumored to be asking Ben Affleck to run for the Senate. Note to the Virginia Democrats: Ben Affleck is an A-C-T-O-R not a politician. Am I the only one who is completely fed up with actors running for (and holding) public office? I mean what exactly are their job-qualifications anyway? Pretending to be someone they are not for a living? Memorizing scripts? Drinking, gambling and carousing? Clowning for the cameras? Hmm.. Come to think of it, maybe actors are ideally qualified to be politicians. But not good politicians.

I’m sorry but why in the world should Ben Affleck be in the Senate? What exactly does he know about being a Senator? Besides, he hooked up with Jennifer Garner — something I, and legions of other Alias fans, will never forgive him for. Is he just jealous of the Governator? Or this all just a publicity stunt for an upcoming Sequel to some horrible J-LO movie in which Senator Ben and his kid’s nanny, J-LO, team up to save the world? Come-on, people, what happened to the concept of real Leaders, you know — statesmen — people who actually have some actual understanding of Government, let alone experience?

We wonder why America is not what it used to be. Well, perhaps it’s because the government is increasingly run by showboating businessmen and Hollywood celebrities with no political experience, instead of real public servants with a lifetime of practical wisdom about how to grow and lead a nation.

Continue reading

Color Laser Printers Secretly Encode Tracking Codes on Printouts

This is quite interesting. It turns out that manufacturers of color laser printers are secretly encoding tracking numbers onto every inch of every printout. These microscopic codes enable printouts to be traced back to particular printers that printed them, and thus to whomever owns those devices. I’m surprised there hasn’t been more discussion of this.

Senate May Ram Ludicrous Copyright Bill

From the Truly-Ludicrous-Department, the Senate may attempt to ram through a horribly shortsighted and backwards copyright bill. This bill would, among other things, eliminate centuries of fair-use precedent, make skipping commercials on digital recording devices a crime, and would make many uses of p2p networks, digital music players, digital sampling, and other forms of fair-use, crimes. The effect of this bill, if it passes, will simply be to force even more of the US technology and online services industry offshore; it will reduce the size of the US Market for emerging media devices; it will reduce our international competitiveness as a technology-leading nation; it will stunt innovation in next-generation media and online services; and it won’t stop anyone from filesharing anyway. What a waste of legislative energy.

Can someone please make a really good, free, open-source, cross-platform tool for making "darknets" for private, secure filesharing with trusted groups of friends? If such a tool were created (and designed to be really secure so that only invited parties can see into it), then that would pretty much end all these silly attempts to prevent filesharing.

The world has changed, but instead of accepting this fact and adapting laws and business models to fit it, the lawyers and legislators are trying to make even more restrictive laws in a futile attempt to turn back the tides of technological and social innovation. It’s never worked in the past and it won’t work this time either. In the age of information, information business models must evolve. You cannot limit information or control it effectively once it is digitized. Instead, business models should be developed that leverage the fact that "information wants to move freely" — the fact that digital information tends to spread faster and more widely than printed or analogue data is a blessing, not a curse. The problems of marketing and distribution are no longer obstacles or cost-centers because people are empowered to transmit information they like to others who might like it too.

So where will the money be made? Instead of charging for information licenses content providers should start to focus on making money from the "halo" around each information object. For example, from related services and transactions around the information.

For example, instead of charging a license fee to download and own a musical recording, why not charge a fee for a service that supplies high-quality recommendations for music to listen to that you are likely to like; or a fee for a service that aggregates all the music and video you are likely to like and makes it available to you on-demand from anywhere via high-bandwidth streaming to your devices? Instead of paying per-item, users would simply pay for access to the library, and/or for use of bandwidth.

Superdistribution, which I have written about elsewhere, is another way to make money from filesharing. Contrary to preventing filesharing, it actually encourages it, because sometimes people downstream will end up buying things as a result. For example, suppose you could download and listen to any song for free, for the earlier of a certain number of times played or a certain number of days. After that date you either must pay or lose your license. However, the price you pay gets lower and lower as the number of further people you pass the music on to increases, and the number of people they pass it on to increases, and so on. And it gets even lower if some of them eventually pay to license the song. In other words, if you spend your labor to market a song you could get a discount or get it for free, or even make a profit. Now that is empowering customers to be product evangelists!

Or another example of a business model: why not start simply asking for optional "tips," payable in micropayment denominations from people when they listen to music they like — people are not averse to supporting bands they really enjoy, just look at how they put money in the cup when listening to a good street musician — as long as they know the money actually goes at least in part to the bands!

Or for example, instead of forcing TV viewers to watch commercials they don’t want to see, why not provide optional commercials that target different types of interests and enable users to opt-in to get them during a TV show, or after it. For example, when watching a sitcom, perhaps a viewer might want to know where to get the clothing, music, furniture, or gadgets featured in the episode. These are just a few ideas for "next business models" that are based on empowering content consumers and allowing information to be unconstrained.

The American Conservative Magazine endorses…Kerry?

Disclaimer: I try to keep this blog non-political. I am essentially a political centrist — I tend to judge each politician, plan and party on its merits rather than by any rigid ideology or party-line. I generally find most politicans, political plans and political parties to be less intelligent and less long-term-oriented than I would wish them to be. As a result, from my perspective, choosing among them is often a matter of choosing the lesser-of-evils. So by posting this article I don’t want to give anyone the impression that I am a partisan person or that this site is partisan. Far from it. In fact, as you can probably see from this site’s content, having an open mind is one of my core values. I also have lots of friends on both sides of every major issue, and they’re all intelligent people whom I respect. There are liberal ideas that make sense to me and also a lot of conservative ideas that make sense to me. May the best memes win!

An article has just come out that is so significant I think it deserves wide exposure. The American Conservative Magazine — the voice of conservatives in America — has officially endorsed John Kerry instead of Bush. Now that’s unprecedented. It shows that there is a tremendous split taking place among conservative voters.

Read it yourself, here. It’s really something. I would be interested in hearing comments on this article from both conservatives and liberals.

More Rumors of Pre-Emptive Iran Strike

This article by a former NSA/NSC and Reagan-administration official adds further credence to the “October Surprise” theory that the White House may initiate a pre-election attack on Iran. It’s interesting because of the source, who is presumably both more reliable and more connected and “in-the-know” than the usual rumor sources on the Web. Let’s hope he’s wrong or that the action is cancelled.

A pre-emptive strike on Iran could quickly spiral out of control and involve other nations — for example Russia. Iran has a huge military and would be a much more serious opponent than Iraq. There is no immenent threat from Iran — there is time to solve the situation through diplomacy. Considering that we are currently losing the war in Iraq and facing severe troop shortages, I don’t see how we could fight another ground war against a country as powerful as Iran.

A further complication is that a limited strike on just the disputed nuclear installation would run a high risk of major retaliation by Iran against the US, Israel, our forces in Iraq and other allies. The only way to protect against that risk would be to NOT do a limited strike but rather to do an instant decapitation attack in order to wipe out any potential retaliatory capabilities. It seems unlikely that such an instant decapitation attack could have a high probability of success without the use of nuclear weapons. And this is what worries me. There is certainly a risk if Iran continues to build up its nuclear capabilities. But is this risk worse than the risks of a nuclear conflict in the region?

The USA is still the first and only nation to ever use nuclear weapons in the field of battle. Let’s hope we don’t use them again. If we do, I think we will certainly have lost all moral-imperative on the world stage. Nuclear weapons should never be used unless there is no alternative, and certainly not pre-emptively. To do so would set a terrible precedent, weakening our case against nuclear proliferation in other nations, and will make the world a more dangerous place.

If Russia and China witness us making a pre-emptive nuclear strike against Iran, why should they not think we might do the same against them? Actions speak louder than words. We have to be very careful and think about future generations, not just in our own country, but in others. While the danger of nuclear weapons being used by terrorists or against Israel is certainly something to combat, if we use those same weapons ourselves (which will inevitably result in mass civilian casualities no matter how careful we are) are we any better than the terrorists?

From a Reader: TV News: A Mirror Reflecting the World It Covers or Making It A Better Place in Which to Live One Story at a Time?

This article was submitted to Minding the Planet by a reader, James Clayton Napier, the author. It’s an interesting inside commentary on how TV News works (or doesn’t work, as the case may be) — NS

“Images can either imprison or liberate us.” –Roberto Assagioli

Feature stories are vanishing from TV news broadcasts. The public speaker, television feature reporter, and movie maker are storytellers. While there is an underlying structure to what the person in each venue does, a structure the audience is often unaware of, there seems to be a trend these days away from content (a central focus) and a movement toward the shock value of an experience—what is said, what is heard, what is seen.

The adrenalin response becomes, in this context, more important than the cognitive ability to process (think about) what one is experiencing.

Continue reading

How to Save the Upcoming Elections from Terrorism Alert Manipulation

There has been much recent discussion lately about alleged evidence that the Bush administration is issuing terrorist alerts for political gain. While I am not taking a position on this issue, I do have a suggestion that could eliminate any doubts, and in the process protect our upcoming elections.

In order to prevent the possibility that national terrorism alerts might be issued for political gain by an incumbent Presidential administration, the right to issue or imply terrorism alerts and the right to postpone elections, should be given to a bi-partisan committee. This policy change should be instituted immediately.

Continue reading

Can You Imagine What Would Happen if MoveOn.Org Used the GoMeme Concept?

I wonder if anyone from MoveOn.Org or the Republicans will notice our GoMeme experiments? (Not that I’m taking sides — I’ll simply be happy if somebody wins the election!) Grassroots political campaigns could potentially really benefit from the techniques we’re testing here. For example, imagine a “blog meme” for a political campaign — a meme that states some useful facts about a candidate and their opponent, perhaps has some survey questions and a GUID, and has the added benefit of a cool Improve-Your-Google-Ranking-By-Hosting-This-Meme candy coating? Wow — it could spread the message to a lot of blogs pretty quickly if done right. That might actually work. But I try to stay out of politics, so I’m not taking sides here or endorsing anyone. If you read this and know the “right people” — feel free to suggest the idea to them.