Nova Spivack - Minding the Planet » Philosophy

The Next Generation of Web Search — Search 3.0

Nova — Sat, 23 May 2009 06:26:15 +0000

The next generation of Web search is coming sooner than expected. And with it we will see several shifts in the way people search, and the way major search engines provide search functionality to consumers.

Web 1.0, the first decade of the Web (1989 – 1999), was characterized by a distinctly desktop-like search paradigm. The overriding idea was that the Web is a collection of documents, not unlike the folder tree on the desktop, that must be searched and ranked hierarchically. Relevancy was considered to be how closely a document matched a given query string.

Web 2.0, the second decade of the Web (1999 – 2009), ushered in the beginnings of a shift towards social search. In particular blogging tools, social bookmarking tools, social networks, social media sites, and microblogging services began to organize the Web around people and their relationships. This added the beginnings of a primitive “web of trust” to the search repertoire, enabling search engines to begin to take the social value of content (as evidences by discussions, ratings, sharing, linking, referrals, etc.) as an additional measurment in the relevancy equation. Those items which were both most relevant on a keyword level, and most relevant in the social graph (closer and/or more popular in the graph), were considered to be more relevant. Thus results could be ranked according to their social value — how many people in the community liked them and current activity level — as
well as by semantic relevancy measures.

In the coming third decade of the Web, Web 3.0 (2009 – 2019), there will be another shift in the search paradigm. This is a shift to from the past to the present, and from the social to the personal.

Established search engines like Google rank results primarily by keyword (semantic) relevancy. Social search engines rank results primarily by activity and social value (Digg, Twine 1.0, etc.). But the new search engines of the Web 3.0 era will also take into account two additional factors when determining relevancy: timeliness, and personalization.

Google returns the same results for everyone. But why should that be the case? In fact, when two different people search for the same information, they may want to get very different kinds of results. Someone who is a novice in a field may want beginner-level information to rank higher in the results than someone who is an expert. There may be a desire to emphasize things that are novel over things that have been seen before, or that have happened in the past — the more timely something is the more relevant it may be as well.

These two themes — present and personal — will define the next great search experience.

To accomplish this, we need to make progress on a number of fronts.

First of all, search engines need better ways to understand what content is, without having to do extensive computation. The best solution for this is to utilize metadata and the methods of the emerging semantic web.

Metadata reduces the need for computation in order to determine what content is about — it makes that explicit and machine-understandable. To the extent that machine-understandable metadata is added or generated for the Web, it will become more precisely searchable and productive for searchers.

This applies especially to the area of the real-time Web, where for example short “tweets” of content contain very little context to support good natural-language processing. There a little metadata can go a long way. In addition, of course metadata makes a dramatic difference in search of the larger non-real-time Web as well.

In addition to metadata, search engines need to modify their algorithms to be more personalized. Instead of a “one-size fits all” ranking for each query, the ranking may differ for different people depending on their varying interests and search histories.

Finally, to provide better search of the present, search has to become more realtime. To this end, rankings need to be developed that surface not only what just happened now, but what happened recently and is also trending upwards and/or of note. Realtime search has to be more than merely listing search results chronologically. There must be effective ways to filter the noise and surface what’s most important effectively. Social graph analysis is a key tool for doing this, but in
addition, powerful statistical analysis and new visualizations may also be required to make a compelling experience.

Nowism — A Theme for the New Era?

Nova — Sat, 23 May 2009 05:52:25 +0000

DRAFT 1 — A Work in Progress

Introduction

Here’s an idea I’ve been thinking about: it’s a concept for a new philosophy, or perhaps just a name for a grassroots philosophy that seems to be emerging on its own. It’s called “Nowism.” The view that now is what’s most important, because now is where one’s life actually happens.

Certainly we have all heard terms like Ram Das’ famous, “Be here now” and we may be familiar with the writings of Eckhart Tolle and his “Power of Now” and others. In addition there was the “Me generation” and the more recent idea of “living in the now.” On the Web there is also now a growing shift towards real-time, what I call the Stream.

These are all examples of the emergence of this trend. But I think these are just the beginnings of this movement — a movement towards a subtle but major shift in the orientation of our civilization’s collective attention. This is a shift towards the now, in every dimension of our lives. Our personal lives, professional lives, in business, in government, in technology, and even in religion and spirituality.

I have a hypothesis that this philosophy — this worldview that the “now” is more important than the past or the future, may come to characterize this new century we are embarking on. If this is true, then it will have profound effects on the direction we go in as a civilization.

It does appear that the world is becoming increasingly now-oriented; more real-time, high-resolution, high-bandwidth. The present moment, the now, is getting increasingly flooded with fast-moving and information-rich streams of content and communication.

As this happens we are increasingly focusing our energy on keeping up with, managing, and making sense of, the now. The now is also effectively getting shorter — in that more happens in less time, making the basic clockrate of the now effectively faster. I’ve written about this elsewhere.

Given that the shift to a civilization that is obsessively focused on the now is occurring, it is not unreasonable to wonder whether this will gradually penetrate into the underlying metaphors and worldviews of coming generations, and how it might manifest as differences from our present-day mindsets.

How might people who live more in the now differ from those who paid more attention to the past, or the future? For example, I would assert that the world in and before the 19th century was focused more on the past than the now or the future. The 20th century was characterized by a shift to focus more on the future than the past or the now. The 21st century will be characterized by a shift in focus onto the now, and away from the past and the future.

How might people who live more in the now think about themselves and the world in coming decades. What are the implications for consumers, marketers, strategists, policymakers, educators?

With this in mind, I’ve attempted to write up what I believe might be the start of a summary of what this emerging worldview of “Nowism” might be like.

It has implications on several levels: social, economic, political, and spiritual.

Nowism Defined

Like Buddhism, Taoism, and other “isms,” Nowism is a view on the nature of reality, with implications for how to live one’s life and how to interpret and relate to the world and other people.

Simply put: Nowism is the philosophy that the span of experience called “now” is fundamental. In other words there is nothing other than now. Life happens in the now. The now is what matters most.

Nowism does not claim to be mutually exclusive with any other religion. It merely claims that all other religions are contained within it’s scope — they, like everything else, take place exclusively within the now, not outside it. In that respect the now, in its actual nature, is fundamentally greater than any other conceivable philosophical or religious system, including even Nowism itself.

Risks of Unawakened Nowism

Nowism is in some ways potentially short-sighted in that there is less emphasis on planning for the future and correspondingly more emphasis on living the present as fully as possible. Instead of making decisions with their effects in the future foremost in mind, the focus is on making the optimal immediate decisions in the context of the present. However, what is optimal in the present may not be optimalover longer spans of time and space.

What may be optimal in the now of a particular individual may not at all be optimal in the nows of other individuals. Nowism can therefore lead to extremely selfish behavior that actually harms others, or it can lead to extremely generous behavior on a scale that far transcends the individual, if one strives to widen their own experience of the now sufficiently.

Very few individuals will ever do the necessary work to develop themselves to the point where their actual experience of now is dramatically wider than average. It is however possible to do this, while quite rare. Such individuals are capable of living exclusively in the now while still always acting with the long-term benefit of both themselves all other beings in mind.

The vast majority of people however will tend towards a more limited and destructive form of Nowism, in which they get lost in deeper forms of consumerism, content and media immersion, hedonism, and conceptualization. Rather than being freed by the now, they will be increasingly imprisoned by it.

This lower form of Nowism — what might be called unawakened Nowism — is characterized by an intense focus on immediate self-gratification, without concern or a sense of responsibility for the consequences of one’s actions on oneself or others in the future. This kind of living in the moment, while potentially extremely fun, tends to end badly for most people. Fortunately most people outgrow this tendency towards extremely unawakened Nowism after graduating college and/or entering the workforce.

Abandoning extremely unawakened Nowist lifestyles doesn’t necessarily result in one realizing any form of awakened Nowism. One might simply remain in a kind of dormant state, sleepwalking through life, not really living fully in the present, not fully experiencing the present in all its potential. To reach this level of higher Nowism, or advanced Nowism, one must either have a direct spontaneous experience of awakening to the deeper qualities of the now, or one must study, practice and work with teachers and friends who can help them to reach such a direct experience of the now.

Benefits of Awakened Nowism: Spiritual and Metaphysical Implications of Nowist Philosophy

In the 21st Century, I believe Nowism may actually become an emerging movement. With it there will come a new conception of the self, and of the divine. The self will be realized to be simultaneously more empty and much vaster than was previously thought. The divine will be understood more directly and with less conceptualization. More people will have spiritual realization this way, because in this more direct approach there is less conceptual material to get caught up in. The experience of now is simply left as it is — as direct and unmediated, unfettered, and unadulterated as possible.

This is a new kind of spirituality perhaps. One in which there is less personification of the divine, and less use of the concept of a personified deity as an excuse or justification for various worldy actions (like wars and laws, for example).

Concepts about the nature of divinity have been used by humans for millenia as tools for various good and bad purposes. But in Nowism, these concepts are completely abandoned. This also means abandoning the notion that there is or is not a divine nature at the core of reality, and each one of us. Nowists do not get caught up in such unresolvable debates. However, at the same time, Nowists do strive for a direct realization of the now — one that is as unmediated and nonconceptual as possible — and that direct realization is considered to BE thedivine nature itself.

Nowism does not assert that nothing exists or that nothing matters. Such views are nihilism not Nowism. Nowism does not assert that what happens is caused or uncaused — such views are those of the materialists and the idealists, not Nowism. Instead Nowism asserts the principles of dependent origination, in which cause and-effect appears to take place, even though it is an illusory process and does not truly exist. On the basis of a relative-level cause-effect process, an ethical system can be founded which seeks to optimize happiness and minimize unhappiness for the greatest number of beings, by adjusting ones actions so as to create causes that lead to increasingly happy effects for oneself and others, increasingly often. Thus the view of Nowism does not lead to hedonism — in fact, anyone who makes a careful study of the now will reach the conclusion that cause and effect operates unfailingly and therefore is a key tool for optimizing happiness in the now.

Advanced Nowists don’t ignore cause-and-effect, in fact quite the contrary: they pay increasingly close attention to cuase-and-effect and their particular actions. The natural result is that they begin to live a life that is both happier and that leads to more happiness for all other beings — at least this is the goal and example of the best-case. The fact that cause-and-effect is in operation, even though it is notfundamentally real, is the root of Nowist ethics. It is precisely the same as the Buddhist conception of the identity of emptiness and dependent-origination.

Numerous principles follow from the core beliefs of Nowism. They include practical guidance for living ones life with a minimum of unnecessary suffering (of oneself as well as others), further principles concerning the nature of reality and the mind, and advanced techniques and principles for reaching greater realizations of the now.

As to the nature of what is taking place right now: from the Nowist perspective, it is beyond concepts, for all concepts, like everything else, appear and disappear like visions or mirages, without ever truly-existing. This corresponds precisely to the Buddhist conception of emptiness.

The scope of the now is unlimited, however for the uninitiated the now is usually considered to be limited to the personal present experience of the individual. Nowist adepts, on the other hand, assert that the scope of the now may be modified (narrowed or widened) through various exercises including meditation, prayer, intense physical activity, art, dance and ritual, drugs, chanting, fasting, etc.

Narrowing the scope of the now is akin to reducing the resolution of present experience. Widening the scope is akin to increasing the resolution. A narrower now is a smaller experience, with less information content. A wider now is a larger experience, with more information content.

Within the context of realizing that now is all there is, one explores carefully and discovers that now does not contain anything findable (such as a self, other, or any entity or fundamental basis for any objective or subjective phenomenon, let alone any nature that could be called “nowness” or the now itself).

In short the now is totally devoid of anything findable whatsoever, although sensory phenomena do continue to appear to arise within it unceasingly. Such phenomena, and the sensory apparatus, body, brain, mind and any conception of self that arises in reaction to them, are all merely illusion-like appearances with no objectively-findable ultimate, fundamental, or independent existence.

This state is not unlike the analogy of a dream in which oneself and all the other places and characters are all equally illusory, or of a completely immersive virtual reality experience that is so convincing one forgets it isn’t real.

Nowism does not assert a divine being or deity, although it also is not mutually exclusive with the existence of one or more such beings. However all such beings are considered to be no more real than any other illusory appearance, such as the appearances of sentient beings, planets, stars, fundamental particles, etc. Any phenomena — whether natural or supernatural — are equally empty of any independent true existince. They are all illusory in nature.

However, Nowists do assert that the nature of the now itself, while completely empty, is in fact the nature of consciousness and what we call life. It cannot be computed, simulated or modeled in an information system, program, machine, or representation of any kind. Any such attempts to represent the now are merely phenomena appearing within the now, not the now itself. The now is fundamentally transcendental in this respect.

The now is not limited to any particular region in space or time, let alone to any individual being’s mind. There is no way to assert there is a single now, or many nows, for no nows are actually findable.

The now is the gap between the past and the future, however, when searched for it cannot really be found, nor can the past or future be found. The past is gone, the future hasn’t happened yet, and the now is infinite, constantly changing, and ungraspable. The entire space-time continuum is in fact within a total all-embracing now, the cosmically extended now that is beyond the limited personalized scope of now we presently think we have. Through practice this can be gradually glimpsed and experienced to greater degrees.

As the now is explored to greater depths, one begins to find that it has astonishing implications. Simultaneously much of the Zen literature — especially the koans — starts to make sense at last.

While Nowism could be said to be a branch of Buddhism, I would actually say it might be the other way arond. Nowism is really the most fundamental, pure, philosophy — stripped of all cultural baggage and historical concepts, and retaining only what is absolutely essential.

Metascience: The Convergence of Science and Religion

Nova — Wed, 15 Apr 2009 23:51:33 +0000

(DRAFT 7. Work-In-Progress)

What is the universe and where does it come from?

There are two major schools of thought on this question:

Science: One is modern-day science, which takes the position that universe is strictly a physical phenomenon and that everything about can be explained by repeatable physical measurements, testable scientific theories, and the rules of math and logic.
Religion:
The other school of thought is religion, which in general, takes the position that the universe comes from something non-physical that is ultimately unexplainable and beyond the reach of science.

In this paper we will take an intellectual adventure into the far fringes of both science and religion, to explore the question of whether or science and religion might be unified. Such a unification is an intellectual “Holy Grail” that could truly change the world. But is it even possible? I think it is, and I’ll propose the core of such a unification here.

The Possibility of Convergence

While there are clearly differences between the approaches and beliefs of the sciences and religions of the world, there are also more similarities than many would like to admit. Beyond that however, at the very deepest levels, they lead to similar logical conclusions and in fact intersect on certain fundamental points, whether their proponents know it or not.

In particular, the question of the origin and nature of the universe is where I believe science and religion converge. Whether one holds the view of science, the view of religion, or both, it turns out that there is a logical necessity for reaching the same final conclusions about the ultimate nature of reality.

Whether one starts from a scientific viewpoint and applies only the methods of science and logic, or one starts from a religious perspective and applies only the methods of religion and logic, either way the conclusion is the same. As long as one regards logic as a valid method of enquiry, the final answer is the same.

The Core Argument

So what is the answer? In short, everything is “nonoriginated.” This has a very specific meaning: the universe (or anything else that we might posit to exist) cannot logically originate from nothingness, from itself, or from some other fundamental thing.

Here’s how this conclusion is reached in a nutshell (I will explain this argument in more depth later in this article, as well as its many implications):

To claim that something originates from nothing is a contradiction.

To claim that something originates from itself is a contradiction.

To claim that something originates from something else leads to an infinite regress unless you claim there is a fundamental first-thing — but claiming there is a fundamental first-thing leads to a contradiction, so it’s not an option. An infinite regress on the other hand, is not really an origin.

Therefore none of the three above ways of originating are logically tenable, yet there is no other possible fourth alternative.

This then leaves only two possible conclusions about the universe (and anything else that is posited to exist):

The first option is that the universe is not really happening at all, because there’s no logical way for it to have originated. But this is immediately contradictory to our experience. It is refuted by obvious, undeniable evidence — right in front of us we can see that something is happening — who knows what it is, but it would be absurd to deny that there is some kind of phenomena taking place.
The second option is that the universe is happening, although there is no origin for it (i.e. it is “nonoriginated”). It is not necessary for there to be an ultimate and final origin — no first cause, prime mover, fundamental particle, or first moment of creation. The universe must therefore be infinite in time, space, and levels of scale.

Option (1) is easily refuted. We are left with option (2) – Nonorigination.

But it is a bit strange to imagine a universe that has no beginning, no origin. How can the universe exist if it is truly beginningless? Without a first-cause what could ever have gotten it started? Without a final fundamental particle, what could things actually be made of? In fact, it is precisely because the universe is nonoriginated that it CAN appear at all. This will be explained further in this article.

We can see how this logic applies to the origin of the universe. How about God? Well if God exists then the same logic would apply: God must also be nonoriginated. Anything that is posited to exist must be nonoriginated.

This point of nonorigination is where science and religion intersect. Nonorigination is the ultimate nature of reality. It is not merely a concept — it is the actual nature of all things, and it has many profound implications. It points to a level of reality that is beyond the limits of space and time — and in this respect it is proof of what might be called the Divine, yet it is also completely compatible with the physical world and its laws.

There are several other key dimensions of nonorigination as well. Awareness is one of them. Awareness is the unique capacity of sentient being to make observations. It plays an important role in making the universe happen, and is actually unified with nonorigination. Where there is nonorigination there MUST be awareness and vice-versa.

Likewise the process of cause-and-effect turns out to be a natural corollary to the nonorigination of the universe, and it’s powered by awareness, the act of making observations. If there were no such process, the universe could not work as it does; it would effectively be random.

I will explore these topics in a lot more detail below.

The unification of science and religion is not philosophy, it is logic. But how we interpret it, and what we do with it is a matter of personal preference and personal philosophy. This paper will not attempt to draw conclusions about what scientific or religious belief is best. That is up to you. Use the logical evidence however you see fit.

What Does the Universe Come From?

If one even merely posits the existence of the universe or even just the presence of a fundamental particle — then that immediately leads to further questions such as: Then where does that come from, what is it all really made of, and how could it all be taking place, what is space-time made of or located in, who or what designed this or how did it all happen so perfectly when it is statistically almost impossible?

Some people just can’t imagine that anything as vast as God could be possible, so they simply decide (without any real evidence) that God is impossible. Or they think that there could not be anything greater than or beyond the scope of the physical universe because they feel that the only things that can exist are physical things. To them, there is nothing but the physical, it is all a big machine, this is all there is — and for that reason they can’t believe in some kind of greater being or ultimate reality beyond space and time or the physical laws. But the grounds on which they claim God is not possible can also be used to claim the universe itself is not possible. If they believe in the possibility of the physical universe they also must accept the possibility of God by the same logic.

Here’s why: If the argument against the possibility of God is that it just isn’t possible for there to be something infinite, then that means either space and time are finite or they can’t exist either — the universe would not be possible because space and time are presently thought to be infinite.

Similarly, if the argument against the possibility of God is that there just couldn’t be anything beyond the physical universe, then even the physical universe could not exist — for if there were no possibility of anything greater than or beyond the universe then where is the physical universe taking place? What does it come from? What is it “in?” If it ever ends, what remains? This second argument is a bit of a difficult point so it bears further explanation.

Whenever you posit something, it logically has to either come from nothing, or from itself, or from something else. And at the time it exists it either has to depend on nothing, depend on itself, or depend on something else.

Stating that the universe comes from nothing or depends on nothing is problematic — it is in fact equivalent to saying that the universe comes from or depends on something beyond the universe: some primordial “nothingness.”

Stating that the universe comes from or depends on itself is circular and also a contradiction of sorts — in order for the universe to create itself or depend on itself it must already exist, and so this is impossible and not an option.

Yet stating that the universe comes from something else or depends on something else admits that there must be something beyond it to come from or depend on.

In other words, no matter what position one takes on the universe, it leaves open the possibility – indeed even the logical requirement – that there must be something before it, greater than it, deeper than it, beyond it, after it, etc.

Refuting Ideas that the Universe Comes from Nothingness

There are however some people who are not convinced by the above arguments. They hold tenaciously to the belief that the universe comes from some kind of primordial “nothingness” which they conceptualize as existing somehow on its own, either before or during the existence of the universe.

This belief in some kind of concrete “nothingness” has many problems. First of all to posit “nothingness” is to treat it as some kind of thing in fact — so it is self-contradictory from the start. Secondly, it is impossible to even imagine actual “nothingness” so labelling it, speaking of it, or positing that it exists is simply delluded. To posit it is not actually to posit it. To imagine it is not actually to imagine it. And in fact there is no way to even conceive of nothingness actually existing, for if it were to exist it would not be nothing. Finally, even if we ignore all these logical problems and still cling to the concept of nothingness, how could anything come from nothing? Let’s examine further.

If nothing really is “nothing” it could not contain anything that serve as a cause or origin for anything else, let alone an entire universe. So it could not give rise to anything. In fact it would be a contradiction to assert the co-existence of nothing and something as well — so even if nothingness could somehow give rise to the universe it would have to be destroyed or eliminated at the moment the universe
came into existence — but if that were the case how could it give rise to the universe — it could never overlap with the universe at all so how could it even be said to give rise to it?

For example the universe could not gradually emerge from nothingness since nothingness would be completely eliminated at the very first instant of the process of emergence, and then the process would be over since there would be no more nothingness left for the rest of it to emerge from.

Similary the universe could not emerge all-at-once from nothingness either, because for that to happen there would at least have to be a moment in which nothing and the universe co-existed — the moment in which the universe emerged.

If we don’t allow for at least that one moment of co-existence before the universe replaces nothingness, then causality is not possible to establish: there would be no way to connect the emergence of the universe as coming out of or from a pure state of nothingness that existed before it — and so there would be no point in making this claim at all.

To say that one thing comes from another thing means we have to be able to show how they are connected, and for that to be possible they have to both exist at the same time, or there has to at least be some chain of events we can point to that connects them.

But if nothing and something are truly mutually exclusive then that is simply not possible to establish. All this effort is simply to show finally and totally that nothingness is a flawed concept, and to claim that something can come from nothingness is even more flawed. If you already accept that you don’t have to re-read this paragraph to figure it out, just continue reading below.

Furthermore belief in the concept of nothingness actually refutes belief in the power of science — for nothingness is not measureable, not verifiable in any way, and is therefore impenetrable to science.

So anyone who cites “nothingness” as the origin of the universe is not in fact being scientific, they are abandoning science. To claim that all space and time — and all science — springs from nothingness is akin to claiming that the physical world (and therefore the domain of science) depends upon something beyond the physical world and beyond domain of science, in other words on what is traditionally the domain of religion.

In other words, if we think the universe sprang forth from nothingness that is like saying that science depends on something beyond the realm of science at the fundamental level, and if we say the opposite — that the universe has always existed or there is an infinite series of universes — that is also akin to saying that science depends on something beyond what science can ever explain — for infinity, while not a contradiction at least, is equally impenetrable to science.

Refuting Ideas that the Universe Comes from Itself

If the universe didn’t spring forth magically from nothingness, then perhaps it came from itself. What would this mean? It would mean that the universe already existed before the universe existed, in other words it both existed and did not exist at the same time. That is circular reasoning, and it’s also a logical contradiction. There’s not much more that needs to be said about this. But I’ll say it anyway, just to make it perfectly clear that this is not an option.

Perhaps we might interpret “coming from itself” in a slightly modified manner. For example, the universe today comes from the early universe, and they are quite different. So saying the universe of today comes from the universe of way-back-when is not saying that the universe today comes from itself literally, it is saying it comes from something else: the early universe. That is certainly one way to wiggle out of the fallacy of something coming from itself, but it just leads to an infinite regress: the fallacy of something originating from something else. The next section explores why that isn’t an option either.

Refuting Ideas that the Universe Comes from Something

If the universe doesn’t come from nothingness, or from itself, then what does it come from? If it comes from something else, then what does that thing come from? At some point there has to be a beginning to the process. But if there is a beginning then what is before it? Whatever that is, it is beyond the realm of science.

To state that the universe comes from something else is to say that something else (whatever it is) is the more fundamental level or prior state of the universe. In other words to state that the universe comes from something is really saying the universe comes from the universe, at a deeper level or an earlier time, or a different place, or in a different state or form, or all of the above.

But all such statements are either claims that the universe, taken as a whole (all states of the universe over all time and space) comes from itself, or at worst it is a circular argument that simply pushes the problem down a level: what does that other more fundamental “something” that the universe depends on come from?

On the other hand, if we claim that the universe is beginningless and unoriginated — then what is the eternity in which this “beginninglessness” is taking place? What created eternity? To posit that there is an eternity “beyond” the universe, or that “contains” the universe (including space and time) is already to state that there is
something beyond the realm of science, something outside the universe. That’s acceptable, however, if we then claim that this “eternity” is some kind of more fundamental thing, we just end up in the same infinite regress as before.

Another possibility might be to claim that eternity and the universe are the same thing. This is to say that the universe is infinite in scope — space and time are boundless and contain all there is. This is either equivalent to the claim that the universe comes from nothing, or from itself. Neither of those options is tenable.

If we posit that eternity comes from nothing that is a contradiction. If it is self-originated, that is circular and also a contradiction. If we say it comes from something else, then what — an infinite series of greater eternities, each containing all the lesser ones, like a Russian doll? Or is there a highest level of eternity and if so, what prevents there from being greater levels of eternity — what causes the boundary to exist and if there is a boundary, what is on the other side of it? This leads to either a contradiction or an infinite regress.

If one claims that the universe contains all space and time, then is the container and what is contained finite or infinite in scope? If it is finite there must be some kind of edge, if it is infinite it implies something so inconceivably vast it is frankly mystical in scope.

In short, if we claim the universe comes from something that leads to circular arguments and contradictions, or an infinite regress. If we’re willing to accept circular arugments and logical contradictions or infinite regresses as satisfactory answers then that is not very different than accepting any other self-justified claims taken on faith, such as those made by religions. In fact, it’s just a kind of religious belief disguised as science. If we are willing to think this way — and most scientists are — then why not also believe in God or other religious ideas as well? It would be hypocritical not to.

It’s important to note that the same logic that refutes notions that the universe comes from nothing, itself, or something else, can also be applied to any claims that there is a God. If there is a God, then like the universe, it also cannot originate from nothing, itself, or something else without leading to logical fallacies. To claim that God came from nothingness is again the something-from-nothing argument that we know does not make sense under logical scrutiny. To claim that God comes from God is circular reasoning and contradictory. To claim that God comes from something greater than God contradicts the very notion of God and/or leads to an infinite regress which just pushes the problem down to deeper levels — where does that infinite regress of ever greater Gods come from then?

Both the universe and the concept of God have the same existential status in fact. Neither one of them has an origin that we can actually find or name without ending up in a logical mess of contradictions and infinite regressions. In this respect they are quite similar.

Nonorigination

If neither any possible universe nor any possible God can be said to come from nothing, itself, or something else, then that leaves only two logical conclusions:

The first option is that these things are not possible and not happening at all since they can’t have originated — however that option is refuted by the fact that at least in the example of the universe, something is obviously and undeniably happening right now. The presence of the universe refutes the notion that it is impossible for something to exist that does not originate from nothing, itself or something else.
The second option is that such things could be possible, but in an “unoriginated” manner. But what does this mean? In short, for something to be “unoriginated” does not mean it is non-existent, it just means that it is not dependent on some initial set of causes and conditions. One way for something to exist in an unoriginated manner is for it to be eternal, or at least beginningless.

Option (1) is refuted by the basic fact that we do observe something happening right now. Option (2) is the only remaining option, and is not refuted in any obvious manner.

But option (2) is mind-bending. How can something beginningless exist? How could it ever have come about if there were never any initial causes or conditions to start it? It’s the primordial chicken-and-the-egg problem.

And this is where things get interesting. Scientific theories claim the universe either has an origin or is unoriginated. Religions also either claim the universe has an origin or is unoriginated.

In the first case, the claim of an origin (such as theories in which the universe started from some physical event before which there was literally nothing, or in which there was nothing and then a Diety appeared and created the universe), we can prove logically that this leads to fallacies (because the origin cannot come from nothing, itself, or something else), so this view is simply wrong, or provisional at best; it’s not a final explanation.

In the second case, the claim of non-orgination, in which the universe is held to be beginningless and possibly endless (for example a never-ending sequence of Big-Bangs and Big-Crunches, or a timelessly existing realm), this begs the question of where did this never-ending sequence come from? How could it have ever started? What is it, what iseternity and what created eternity?

In either case however, whether we use science or religion to approach the problem of the origin of the universe, we end up at the same place in the end. The path we may travel to get there is different, and certainly the language with which we express the conclusions is quite different, but the final result is the same. Logically speaking, the universe must be either unoriginated or created by something unoriginated. It is the only logically tenable conclusion.

In other words whether universe is thought of as purely physical, or originating from God, the only logically tenable conclusion is that it is nonoriginated. And the same goes for God. We may believe that God is greater than the universe, in other words prior to it, and in this case God and the universe are not equivalent, however, upon final analysis, even in this configuration, the only logically tenable conclusion is nonorigination.

For example, if the universe is a physical thing that was created by God, yet God is nonoriginated, then by inference the universe is also ultimately nonoriginated (via God’s nonorigination). Although provisionally we can state that the universe originates from God, since God is in this case nonoriginated, the universe is ultimately nonoriginated, for no final origin can be found or logically established.

In summary, nonorigination is the single fundamental truth of both science and religion. It is where they converge.

Unification

And now, based on the above lines of reasoning, the final capstone on the argument.

If we posit that only the physical universe exists, then we have no other choice but to say the universe itself must be unoriginated, in other words, uncaused and unconditioned — neither coming from nothing or from something else.

There is no escape from this logical conclusion. Nonorigination is always found to be the ultimate nature of whatever is positied to exist. It doesn’t matter how many levels of reality you think there are, as soon as you posit even one, it’s “turtles all the way down,” to quote the famous expression. In other words, if you posit the universe resting on the back of something (for example, a giant turtle) then that something must in turn rest on the back of something else (another giant turtle, for example), and so on, endlessly. The only way to not have an endless pile of turtles resting on still deeper turtles is to posit a final fundamental turtle, but that makes no sense — for that turtle would be in free-fall, meaning the entire stack of turtles would have no foundation and would topple over. What nonorigination really means however is that the stack of turtles can be infinite or finite – it really doesn’t matter and is equivalent — either way the entire stack itself, whether just 1 turtle our countless turtles, is nonoriginated. This is not to say that the stack depends on something else we call nonorigination, it is to say that the stack itself IS nonorigination.

This is very hard to accept conceptually, but it is a logical conclusion. The only way to deal with it intellectually, once you derive it and are convinced there is no way around it, is to simply accept it. The universe really is beyond conception — it really cannot ever be conceived. It’s infinite and its nature is inconceivable.

Now what’s interesting, and unifying, about this conclusion is that nonorigination is a logical and scientific kind of conclusion, and yet there is something about it that is inconceivable and wondrous like what we think of when we speak of something Divine. Nonorigination is unexplainable, inconceivable, prior to all space and time, beyond the limits of the mind, and the nature of all things. This is at once scientific and Divine — it is something infinitely beyond all conceptual limits — it is the point where everything converges.

Nonorigination is also a very subtle truth, because it neither asserts or refutes the universe and/or the Divine. In fact, what appears is free to appear and function — yet if we analyze it we find it is nonoriginated. That doesn’t mean there are no causes and effects in operation, it doesn’t mean the universe is random — in fact quite the contrary will be shown later in this article.

Nonorigination says nothing about the day-to-day “relative level of the world” and how it functions — it is a statement about the ultimate nature of everything: the originlessness and fundamental essencelessness of whatever appears. Thus when speaking of nonorigination, we can make a conceptual distinction between the relative and ultimate levels of truth. They are both true, one does not contradict the other.

Relative truth is truth within limits — specifically a statement that holds true locally but not globally. Ultimate truth applies globally. In this case within the reference frame of the universe alone, we can say that any effect we observe is originated from various causes and conditions, but within the larger frame of the origin of the entire universe, it is nonoriginated. In any case, whether one chooses to accept this modal logic or not is a matter of personal preference.

Beyond Four Logical Extremes

In Buddhism the ultimate nonoriginated, uncaused and unconditioned primordial nature of reality is said to be “unborn.” Since it has no cause it is never actually created or “born” as some thing, yet since it is also not literal nothingness, it is not entirely non-existent, for if it were nothingness it could not be something that we could even apply the labels of nonoriginated, uncaused and unconditioned to.

That which is nonoriginated is entirely free of all logical extremes:

Existence
Non-existence
Both existence and non-existence
Neither existence nor non-existence

It doesn’t exist because it is not originated. It doesn’t not-exist because it isn’t literally nothingness. It doesn’t both exist and not-exist because that is a logical contradiction.

The fourth logical extreme is the hardest to overcome and there are a few different arguments to conquer it. First of all the assertion of something neither existing nor not-existing is also a contradiction, via double negatives: if it doesn’t exist then this is equivalet to not-existing, and if it doesn’t not-exist then this is equivalent to existing.

Another way to refute this extreme is by the fact that there is no other alternative to existing or not-existing: to exist is to be something, whereas to not-exist is to not be something. How could there be “something” which is neither something or not-something. If it is “something” that contradicts the prong of claim that it is neither “something” or not-something. Yet if it is “not something” then that contradicts the prong of the claim that it is neither something or “not-something.” In other words, to claim that something is neither something or not-something is contradictory from the very start.

The Nonorigination of Nonorigination

It’s important not to get stuck on conceiving of nonorigination as a special kind of thing. Nonorigination is in fact nonoriginated too. So it can’t be something. It also can’t be nothing. It’s actually free of of four logical extremes of being something or nothing. It’s not any of these four logical possibilities:

Something
Nothing
Something and nothing
Neither something nor nothing

There are no other logical possibilities than these four. Nonorigination cannot be said to be or not to be.

In fact, if we look for nonorigination we don’t find it. For example, you cannot find the absence of something. The absence of that thing is literally the fact that you cannot find it. Nonorigination is the absence — in any moment of experience — of anything that can be found to exist, not-exist, exist and not-exist, neither exist nor not-exist. It is an absence, not the presence of something else that could be labelled “nonorigination.”

But this absence is not merely a rhetorical or logical point — it really is the actual fundamental nature of reality. In other words, whatever the universe is — whatever appears to us — really does have this nature of nonorigination, this complete absence of existing, not-existing, both, or neither. This means the universe is far more unexplainable than can even be imagined.

The Primordial Nature of Reality

We have found that whatever there is, it must be nonoriginated. There is no other logical alternative. Even nonorigination is nonoriginated. So while there is no final isolated thing we can point to as nonorigination itself, the fact that whatever we can point to is always found to have a nature of being nonoriginated is a fundamental truth. In fact it is perhaps the fundamental truth. It’s the one logical conclusion that we always reach no matter what we analyze. All roads lead to nonorigination.

If we say that the universe is nonoriginated, then it doesn’t exist the way that most scientists and even most religious thinkers imagine it to. While it’s not nothingness, it’s also not something, or any other alternative. This absence of having an existential status is in fact the way it really is, that’s its primordial and ultimate nature. We can also say that this absence of existential status is the primordial nature of reality.

This means that reality is beyond the limits of existing and non-existing. This may defy common sense, or even feel impossible to imagine, yet it is the only logical option — it is inconceivable yet must be so.

Many great religions all agree on this point at their highest levels of philosophy: Buddhism, Christianity, Judaism, Islam and Hinduism all agree at the purest conception of the Divine is really inconceivable and unameable, and certainly primordial (not created or conditioned by anything else). At it’s very purest essence the universal truth of all religions, and even of science, is that there must be, and is, something uncreated and unconditioned at the root of reality.

Whether the universe is theorized to have sprung out of perfect randomness or nothingness, or it is an eternity, or there are infinite parallel universes, the only logically tenable way that the entire reference frame can exist is if it is nonoriginated. This nonoriginated, uncaused and unconditioned nature, is the primordial nature of reality — of the universe and/or the Divine — regardless of whether one believes in just one, or in both.

So there we have it: the essence of the universe and the essence of the Divine are the same primordial unoriginated reality. We can call that the universe, we can call it God, or we call it Buddha, Christ, Allah, Tao, or something else. It doesn’t matter what we call it really, it is nameless.

Freedom

If something is truly nonoriginated, in other words, uncaused and uncreated, then it is totally free. In particular it is free of all concepts and beliefs about it or anything else. It is free of all limitations. We cannot say that it has a particular name and no other name. We cannot say it can only be reached through one path and not others. We cannot say that it can only be served by obeying particular rules and not others. We cannot say that only some people have access to it while others don’t, or that anyone is closer to it than anyone else.

Who are we to say anything that would limit something that is totally uncaused and unconditioned? Something cannot be partially free. Either it is totally free or it is not free at all. There is no middle ground. If we truly believe in a conception of a “God” that is totally free, then we have to be careful not to impose further concepts onto it or onto ourselves or anyone else. The closer one is to knowing God, the less one can really say about God.

The same goes for science: we eventually must reach similar conclusions about the fabric of reality and the origin of the universe. We may be able to describe and predict all sorts of things about the physical universe, but the deeper or farther we look in space and time, the more it starts to become indescribable. At the smallest scales and the largest scales, and in fact at every scale in between, the origin and nature of the cosmos is and will always be a mystery. The best we can do is categorize it and glean some understandings about how it functions, but we’ll never be able to explain it. The universe, like God, is also beyond conception. It is either uncaused and unconditioned itself — which means it is free — or it depends on something that is uncaused and unconditioned. Either way, it is free.

Think about that for a moment. If the universe is free or depends on something that is free — then either way, what takes place in the universe is ultimately uncaused and unconditioned, meaning the universe is effectively free in both cases. What does “free” actually mean? It means literally that anything can happen. Anything. Any universe is possible. Any set of physical laws are possible. Anything at all is possible — even things which we can’t explain and which perhaps are contradictory to the physical laws (such as anomalies, miracles, etc.).

Observation

But then why do only particular things appear to happen, rather than other alternatives? Why does the universe appear to obey particular physical laws? Why don’t we observe miracles or other anomalies that contradict the physical laws (note: some people do claim they observe these phenomena, so we cannot say with certainty that they don’t happen at all…)? But in any case, why does the universe seem so rational and orderly if indeed absolutely anything is possible?

One school of thought on this question (the Many Worlds interpretation of quantum mechanics) answers that in fact everything does happen, but in parallel universes, all at once. So there’s no real choice being made — all possibilities from those that are consistent with the universe we know to those which are totally outlandish or seemingly impossible do happen, all at once.

Another school of thought claims that somehow the universe makes choices and that these choices come about whenever observations take place, and that they have something to do with probability — the universe is not precisely deterministic, but not entirely non-determinstic either. If that is the case, then the act of observing something essentially causes the universe to choose what actually happens from the set of all the things that could possibly happen.

But if the universe makes quantum mechanical choices at each moment of observation, then what comes first, the act of observation, or what is observed? What creates reality, what causes the choice that selects one possibility versus all the others? Is what appears literally caused by the observer, or is it there before being observed — does it cause the observer to observer it, or does the observer cause it to be observed? It’s unclear, according to quantum mechanics at least; It’s a
chicken-and-the-egg kind of problem. In fact, the situation is better characterized as a kind of feedback loop, or a dance of sorts, that’s been going on forever.

The universe is ultimately free; anything can happen. But anything does not appear to happen, only some things happen. This is currently said to happen because of choices that are made when observations take place, at least on a subatomic level.

But while observation may cause or condition reality on the quantum scale, on the macroscopic level — the level of people and cars and houses and trees, and so forth — the act of observation does not seem to function in the same manner; it doesn’t cause things to happen. Or does it? The classic Zen koan, “If a tree falls in the woods and there is nobody there to hear it, does it make a sound?” addresses this question.

In fact, if there is no observer to hear the sound, how can we say there is a sound? When the tree falls it causes vibrations, but those vibrations only make a sound if they move the eardrum of something that can hear. If there is no observer, but only a recording device in the woods, there is a recording, but not yet a sound. The sound only can be said to exist when the recording device is actually used to play the recorded sound to an observer. Until that happens, the sound is not observed.

Quantum Mechanics

This strange fact is reflected in scientific experiments such as the famous “Double Slit Experiment” and many variations. In that experiment, the act of measuring the path that a photon takes causes it to appear to appear to behave like a particle, while if you don’t measure the path it appears to behave like a wave. In fact, this effect is even stranger — experiments have been done which seem to indicate that this effect can even go backwards in time. Even if you wait to measure the path the photon takes long after it has traveled through the experiment, that observation seems to effectively go backwards in time and cause the photon to retroactively behave one way or another, in the past.

Another famous thought-experiment which illustrates the interaction between observation and reality is the “Schroedinger’s Cat” example, in which a cat in a box is either dead or alive depending on whether a random event happens, but until you actually open the box you can’t know it’s actual status — and on a quantum level in fact, until the cat is observed you cannot really say it is either dead or alive; it exists in a kind of intermediate state. The moment of observation somehow causes the intermediate state to collapse into a particular quantum state. This is very odd stuff. And for a while it was thought to really only apply at very small scales, although more recently there is some evidence that similar logic may apply even at macroscopic scales.

What this all means is that there is something about observation that seems to cause the universe to make choices. Another way of expressing this is that the universe — because it is totally free — has the freedom to make choices, and this happens through the act of observation. This would also imply that the universe is intelligent and creative, because the things that make observations (sentient beings like humans, for example) are intelligent and creative. Perhaps the universe isn’t happening out there on it’s own, perhaps it is in a very real sense, imagining itself through an unfolding process of creatively making observations.

The Improbability of the Universe

If the universe either is something totally free, or depends on something totally free, then either way, the universe is totally free. That is to say there are no limitations on it. Anything can happen. How then is it that we observe particular things and not everything happening? Why don’t each of us experience all possible parallel universes? Why is the universe the way it is, and not even slightly different? Why are things the way they are? We can look at physical things and use scientific knowledge to understand their trajectories and dynamics. That certainly helps us explain a little bit about those physical things. But it doesn’t tell us why the initial conditions were not different, or why the universe is such that the physical laws and physical constants are what they are.

Even a slight change in the structure or unfolding of the universe would have resulted in a vastly different outcome — the physical laws would be different, the physical constants would have different values, and this would result in different kinds of universes. Some would have very different properties than the one we live in. Some would support life, some would not. Some would have led to our planet and human beings, some would not. Some would have stars and galaxies, yet other extreme cases would burn out and collapse into giant black holes almost immediately, while other configurations would have led to the universe breaking into countless separate universes or literally exploding and then dissolving into countless separate black holes. And there are many other possibilities too. These claims may sound wild, but in fact they are predicted using our current scientific model — if we simply change the initial conditions of the early universe slightly.

So why did things turn out the way they did? And why does our universe seem perfectly balanced to support human life — or any life for that matter? There are so many possibilities for how the universe might have unfolded, and most of those possibilities do not result in a universe that could support human life at all. In fact the universe we live in is one of the more statistically improbable outcomes. The odds of our universe happening are infitessimally small. So how did it happen?

Furthermore, at least on a quantum level it appears that until an act of observation takes place we cannot really say the universe makes a choice about what happens. So what about the early universe — before there were any human observers, or any living things at all to make observations? So what was made the first observation? Was there a “prime observer” at the first instant of the universe, and if not, how could it have come into being since on a quantum level without being observed it could not have had a particular state.

Or alternatively was there some other kind of outside observer that made the original observations of every ancient quantum interaction, enabling the universe to make choices, at least until living observers could evolve to make their own observations? Or, has the universe effectively made all those choices retroactively — for example, now that there are observers, has the effect of our present choices gone back in time and caused the universe to make all the necessary past choices to lead to the way things are today (that one is a mind-bender, but on a quantum level it is not unreasonable or impossible to consider — space and time are not obstacles on the quantum level. For more on this, read about the Anthropic Principle in physics and cosmology)

Perhaps only universes that can support life can therefore contain observers, and so only such universes can actually happen because without observers quantum level choices cannot be made — in other words, possible universes that don’t contain observers effectively cancel themselves out and never even happen, leaving only those universes that can and do support observers. This would at least eliminate a lot of possible universes and improve the odds of universes like ours ever happening. But there are still innumerable, literally countless, variations that are possible even within that set of observer-friendly universes. Why did it turn out that exactly one and only one of those possible universes — ours — is what happened?

Here’s another question that we have to consider as well: If observation is required for the universe to make choices and effectively collapse on various states out of the space of possible states it could be in, then either there was a first observer (which leads the contradiction that the first observer could not happen because it was not observed) or there has to be an infinite regression of observers, or we couldn’t have the present universe at all. Once again, we come to the logical problems we encountered earlier when discussing the universe and God. Either we end up in contradictions or regressions.

One possibility is that the universe is an observer of itself. We know that since the universe can contain observers (for example, humans), it is capable of making observations. So why should observations only happen on the human-scale. Perhaps there are larger systems within the universe that can make observations too? But even if we believe this it still doesn’t solve the problem — even if the universe can observe itself, what observes the universe? Alternatively, if we posit some kind of outside observer of the universe, then again, what observes that? In either case, we end up with a logical contradiction or an infinite regression.

Is there any way out?

Yes, there is one, and only one, way out: It all comes down to consciousness.

Conscious Awareness

Just as we found that in order for the universe to exist either it must be nonoriginated, it also must be inherently observed. Without observation, nothing could happen, choices could not be made, at least according to quantum physics.

But if this the case, what made the first observation that started it all? The answer is that there was no first observation. Instead, observation must be inherently unified with nonorigination. There is no other alternative, at least if observation is necessary for the universe to exist, on a quantum mechanical level.

In other words, the universe does not require an outside observer. This MUST be the case, for on a quantum level the early universe — indeed even the Big Bang or whatever we think the universe was like as far back as possible — could not have happened at all without something observing it (on a quantum level). The capacity to make observations must be an inherent property of the universe itself, or at least of what the universe depends on if we think it depends on something else. Either way, the capacity to observe is inherent, it doesn’t come from nothing, itself, or something else — it has no origin. It has to be or we couldn’t have the universe at all, according to current scientific theories about quantum physics.

So what is this mysterious capacity to observe? It seems to be pretty close to what we mean when we use the terms “consciousness” or “awareness” (and of “God” too by the way).

We humans have this capacity to experience our minds and senses — to not only be aware but to be reflexively aware as well — and it appears that animals and other forms of sentient life have this capacity too. We are able to observe and react to stimulus, but also to know it. We don’t just react automatically, like springs bouncing back from being compressed. We experience what we observe — we know — we are. We have a sense of our own being, we are aware that we are aware. We are aware that we are. And that is observation in its most naked form.

The universe supports the evolution of things which are aware of their own being. And that means that the awareness of being either comes from the physical universe or from beyond it. But either way, we have seen in our earlier discussion, that at the end of the day, whether you believe in only the physical universe or you believe in a God beyond the universe, they have the same ultimate nature of
nonorigination.

The characteristics of the universe, and therefore of what we call “God,” are therefore that of being uncaused, unconditioned AND aware (in other words, making observations). There is no other logical, or scientific, alternative.

Consciousness is therefore something deeper than what we might think. It is a reflection of the universe’s and/or God’s inherent capacity to be aware. It literally IS the primordial awareness of the universe. And because consciousness IS primordial awareness — the basic capacity to make observations that observes at least itself and can potentially observe anything or everything else — that means it is coming directly from the most fundamental level of reality — in fact it IS the most fundamental level of reality.

Awareness is uncaused, unconditioned and aware of being. Each of us, and indeed, each sentient being that is aware of anything, is a reflection of the entire universe in a sense, and of whatever we call “God,” if we believe in God. In a very real sense — from a scientific perspective as well as a religious one — there is something divine in every sentient being, and indeed in the entire universe.

This primordial awareness is inconceivable, because it literally IS that which is nonoriginated. Even within our own minds we cannot describe it or limit it in any way. It is the nature of mind, and it is the nature of reality, and of whatever we might call God. The difference between each of our individual human awarenesses and the infinite and inconceivable awareness of the universe and/or God is one of scale, not one of qualities. This also means that each individual’s mind is potentially as totally free as the total freedom of the universe and/or God. This is our true condition, whether we know it or not. Total freedom means the mind is potentially unlimited — truly unlimited. That means it is possible to know or experience or observe anything, for us as individual sentient beings, and for the universe as a whole.

Although anything can happen in theory, sentient beings such as ourselves and others make observations — that is our function in the universe in fact – and these observations have quantum level repurcussions that actually cause the universe to choose particular outcomes, which in turn feedback to affect the probabilities of our future observations. In a very real sense, observation creates experience.

Whether you believe the universe is an inconceivably vast intelligent and creative being that has free will, or you believe it all depends on a God that is inconceivabley vast, intelligent, creative, and has free will — it’s the same. Take your pick, they lead
to the same conclusion, and the same universe. Awareness — the essence of consciousness — has a very key role in the universe, and/or in whatever we think of as God. It is in fact THE key to it all.

Cause and Effect

From this discussion so far, we have concluded that the universe is nonoriginated. That is to say, the only logical option is that it exists in a nonoriginated manner — it does not arise from nothing, itself, or something else (OR if it arises from something else then that thing must be nonoriginated, or at least something at some point that is causally upstream from it has to be nonoriginated). For example if the universe comes from God, then either God must be nonoriginated, or that which God depends on has to be nonoriginated, and so on. The point is that the series of things and things that create them is finite, not infinite. There is no infinite regress.

This does not deny the operation of cause and effect within the universe, nor does it deny that there can be an infinite series of causes and effects that lead to or stem from any event within the universe. It only denies that there can be an infinite series of causes and effects the lead to the creation of the universe as-a-whole. In
other words, on the relative level, within the universe, cause and effect can operate just as science (or even various religions) might predict. However, the universe as-a-whole is not caused, or eventually depends on something that is not caused.

Therefore the universe as we know it is not contradicted by claiming that it is nonoriginated. Nor is cause and effect contradicted by stating that ultimately the universe as-a-whole, or whatever is that which is nonoriginated, is totally and complely uncaused, unconditioned and therefore free. Furthermore, even though observers — individual sentient beings — within the universe are expressions of that
primordial freedom (by virtue of being aware), they are still subject to the laws of cause and effect within the universe.

For example, a particular observer may make an observation, and in doing so they perturb the universe on a quantum level, which conditions what they end up observing. Observation is a cause. What is observed is partially an effect of the act of observation, and partially an effect of other causes and conditions that relate to it. When an observer makes an observation, together with the appropriate set of causes and conditions, a particular event is observed to take place. Similarly,
that event then acts as a cause or condition for other observations and events to take place for that observer and/or other observers.

In this manner everything that happens within the universe is the result of a complex network of causes and conditions, in which observers play critical roles. Observers actually change the topology of the network (the patterns of linkages between various causes and conditions and observers) whenever they make observations. This ability to rewire the network by making observations is something that is unique to sentient beings — only true observers that are conscious are capable of causing this to happen.

In fact, without observers actively making observations we cannot truly say the network exists in any particular state — it could be in any of an infinite number of possible configurations representing any of an infinite number of possible timelines of universes. The act of observation is what triggers chains of cause and effect to “fire” (almost as if they were patterns of neurons and dendrites in the brain firing sequentially to generate various thoughts). When there is no observation taking place we might say that the universe is frozen in a kind of indeterminate state. Only when observations happen are particular chains of potential cause and effect in time and space activated, and thus particular events they bring about appear to take place.

The process of cause-and-effect changes the probabilities of various events, making them more or less likely to take place, that is, to be observed. And it is the act of observation itself which triggers the chain of cause and effect, which powers it, which makes it happen. This is how the universe works on a quantum level, and also perhaps how it works on other levels too (for example, the law of Karma in Buddhism is effectively this very process of cause and effect, or what is also called dependent-arising, taking place not only in the external physical world and the body, but within all sensory modalities and even within the mind).

But is cause-and-effect required for the universe to function the way it does? Is there an alternative?

Suppose that there were no cause-and-effect within the universe. Instead imagine what it would be like if everything happened randomly. In a totally random universe every event has an equal chance of happening, so either all events would happen at once, or none of them would. We don’t see either of these taking place however. Instead we see very non-random distributions of events taking place.

When you exert a force on an object it is highly likely to exert and equal and opposite reaction on you, and it is quite unlikely that it will do the opposite of that. But in a random universe both events would be equally likely, at least over all time and space and observers and possible universes. So if the all events are equally likely then we could not have the universe we experience, in which that is certainly
not the case.

One might move the problem down a level however by suggesting that perhaps this universe is only one universe in an infinite number of parallel or possible universes, which are all equally likely to happen, and we just got lucky somehow. We happen to be observers within this one, where things fall towards the force of gravity rather than being repelled by it, and so we are able to stand here on the planet and the planet retains its atmosphere, etc.

It’s fine to hold that view, however, even if one does, within this universe at least, it appears to be as if cause and effect is in operation. Whether cause and effect sequences are really happening sequentially over time and are influenced by the free will of observers, or they all happen all at once from the perspective of eternity and thus free will is illusory, what we experience would be the same. Thus these two alternatives are equivalent.

In this universe — which is the only one we observe — it appears to us as if cause and effect processes are unfolding over time, and for all intents and purposes, from our perspectives, whether causality unfolds creatively and non-deterministically over time and in part due to the free will of observers like ourselves influencing what we observe, or it’s all preordained in eternity, its equivalent.

What this means is that for this universe to happen, cause and effect is necessary. There may be other possible universe in the set of all possibilities which may not appear to contain processes that resemble cause and effect, but we are not experiencing any of them right now, nor can we even prove they exist. So from our perspectives it is as if they do not exist. Notably however, we cannot prove they do not exist either.

Now the question is how can a universe that appears to operate by cause and effect, within it, be nonoriginated? How could a universe full of causes and effects not have a cause? How can nonorigination and cause-and-effect be compatible? Isn’t that equivalent to claiming it is an effect (the univeres) that has no cause (nonorigination), and isn’t that therefore a logical contradiction? No. To make such a claim would indeed be a logical contradiction — an effect is the result of a cause and cannot exist without a corresponding cause. The solution is to not claim that the universe is an effect, nor to claim that nonorigination is a cause.

It is contradictory to assert the existence of an effect apart from its cause. Therefore the universe cannot be asserted to be an effect that has no cause. It is simply nonoriginated, it is not the result of anything. For it to be the result of something would contradict nonorigination, which we have already found is the only logical way that the universe can exist at all (because it can’t come from nothing, itself, or something else, so therefore it must either not exist at all, or it must exist in a nonorignated manner, and since it does appear to exist, it must exist in a nonoriginated manner).

Nonorigination requires that the entire universe is not a cause nor an effect. But although the entire universe is not a cause or an effect, it can appear to contain what look like, and function within it as, causes and effects — sequences of events that are causally linked over time and space in complex interdependent networks. This is a real mind-bender and will take some time to explain. Cause-and-effect is a
relative level process — it is provisionally true — but on an ultimate level the process and everything within it is nonoriginated.

For example, we probe further, into any particular event, and we trace back its origins within the universe, and if space and time are infinite, then we may find an infinitely broad and deep network of causes and effects both upstream (leading to it) and downstream (stemming from it) in time. Since these sequences are infinite, they are from a logical perspective infinite regressions. To claim that any effect comes from an infinite series of causes and effects, is logically fallacious — we cannot prove such a claim since we cannot test infinity to see whether or not the series is truly infinite or not, or even what all the causes and effects in the alleged series even are.

Cause and Effect is Nonorigination

Therefore, from a logical level, even though causes and effects may appear within an infinite universe, they too must be nonoriginated — it is the only manner in which they can be said to exist without commiting a fallacy: They must exist in a manner that is free from four logical extremes. In other words, they cannot exist, not-exist, both exist and not-exist, or neither exist or not-exist.

They cannot exist because of infinite regression. They cannot not-exist because that is a logical contradiction and also conflicts with what we observe. Combining existing and not-existing is a logical contradiction. Rejecting both existing and not-existing leads to logical contradiction and also conflicts with what we observe. So while on a relative level the process cause-and-effect appears to operate, on
the ultimate level of analysis, it is equivalent to being unoriginated, from our perspectives at least.

Another way of expressing the same thing is end result is that if the space and time are infinite, then the universe as well as its contents (including all causes, effects, observations, and observers) must be ultimately nonoriginated. And since it’s not possible to have a finite sequence of causes-and-effects (because that would mean that at least one cause or effect would not have a corresponding effect or case, which is not possible (because a cause and an effect are inseperable, it is a contradiction to claim you have one without the other), a finite universe of causes and effects is impossible. Therefore finite universes are impossible, since only universes that contain causes and effects would not be random.

Therefore our universe must be infinite, because we do observe processes of cause and effect, and it also must be nonoriginated (or be equivalent to something that is nonoriginated — for example be being part of an infinite series of causes and effects of universes or by being created by some kind of God’s free will, not by cause and effect (where God is by definition not orignated by anything else). These are the only logical possibilities.

The lines of reasoning in this section, and those above it, prove that lead us to conclude that only infinite universes in which cause and effect appear to operate are possible, and that such universes (and the causes and effects they contain) must be ultimately nonoriginated, and observed, in order to be said to occur.

In other words, cause and effect is nonorigination. Whatever appears to be generated by causes and effects is ultimately nonoriginated.

Nonorigination is Cause and Effect

The same is true in the reverse direction. We cannot say that something is nonoriginated unless there is some relative-level appearance of a thing to make that statement about. The notion that nonorigination could exist on it’s own without some subject or object that is nonoriginated is a contradiction. Nonorigination is a phenomenon that requires a complementary relative-level facet, namely whatever is being asserted to be nonoriginated. To assert nonorigination apart from anything else would be like positing a penny with no sides. A penny must have a heads and tails. It can’t be a penny without them.

Therefore where there is cause and effect is the result of nonorigination and observation, and where there is nonorigination and observation there is some phenomena — some event appearing to take place, and since phenomena do not happen randomly, the only alternative is that some combination causes and effects are at work.

It is the process of observations, causes and effects that makes some possible phenomena more or less likely than others at various locations in space and time. Without such a process all possible phenomena would be equally likely at all possible locations in space and time. That would not result in our universe, or anything like our universe, at least as far as we observers can know from our positions within space and time.

Perhaps one might argue that maybe if we could see eternity we might find that our universe was randomly generated as-a-whole, but that is not possible either — for if all universes were equally likely then they would either all happen at once or none of them would happen at all. The fact that this universe appears refutes the possibility that none of them happen at least. As for the possibility of them all
happening at once, this is a possibility, but we can’t determine this for sure unless we can see eternity ourselves. From our perspective, and as far as we can know, only this one is happening.

Nonorigination is therefore equivalent to cause and effect, and vice-versa. The process of cause-and-effect is not refuted by nonorigination, indeed it is required by nonorigination, and vice-versa. The proof is that this universe is appearing and
functioning the way it does.

Trinity

At each moment of our lives, of each moment of observation no matter how brief or precise — there is something else taking place that is NOT nothingness and NOT exactly whatever appears to us either.

For example when we observe a tree, we see the appearance of the tree visually. That appearance is there, at least as a mere visual image, not unlike an image in a dream. It may be a real image of a real tree, or a dream image of a dream tree — but that doesn’t matter, the two cases are equivalent for in fact we really cannot tell the
difference at the moment of its appearance.

The image of the tree before us is of some thing which we may believe exists “out there” in the “real world” beyond our body and mind, and that it is really just a depiction of the object out there in the visual spectrum, formed by our particular sense organs and their abilities and limitations, and then rendered via the circuitry of our brains onto some kind of internal viewing screen, or to some further set of cognitive processes which then do things like interpret it, label it as a “tree” etc. That’s all fine — whether or not any of that is really what is taking place or not — at the very moment of an appearance appearing that is all hypothetical from our own perspective. All we can know at the moment of an appearance is that it is there in
its own unique way, and that we know it.

The appearance is the object side of a moment of experience. The “we know it” part of the experience is the subject side. There are these two sides to every ordinary moment of experience. This is consciousness, a dualistic interpretation of what is taking place in every moment into having two poles of subject and object that are
somehow two different things. Most people spend their lives experiencing everything — themselves, the outside world, others — in this dualistic mode of cognition. Note that dualism is not inherent, it is a conceptual interpretation of raw experience. Experience itself is not dualistic — there is no actual boundary that we can find between subject and object and we cannot separate them to have one without the other. This dualistic frame of mind is a deep-seated habit and unquestioned belief that is part of our “filter” of the world. It prevents us from knowing experience the way it actually is, and instead splits it like a prism splits a single beam of light, into multiple beams of “subject” and “object” halves of each moment.

It’s key to notice that the dualistic frame of mind — ordinary consciousness — is a kind of artificial division of the moment into two parts. It comes about because a misunderstanding on our own part of what is actually taking place in each moment. What we call the object side of experience is any appearance in any sensory modality or the mind. The subject side of experience is the label we give to the part
of the moment that seems to be witnessing it, or being it.

In fact there are not really two things like this, divided and separate from one another. Instead there is only one thing taking place that has both of these aspects. What is taking place is nonorigination. It has two aspects: awareness and appearance. Actually this triad can be expressed in three formulas:

Nonorigination = awareness + appearance (N = A + A’)

Appearance = Nonorigination – awareness (A = N – A’)

Awareness = Nonorigination – appearance (A = N – A’)

Each moment of experience combines all three of these together into a trinity — they are unified yet still distinct. This might in fact be The Ultimate Trinity of all trinities. Furthermore, if we focus on appearance we will find that it is nonorigination. If we focus on awareness we will find that it too is nonorigination. If we try to focus on nonorigination itself we never find it, instead we always find
moments of awareness plus appearance. Yet if we then try to find the awareness or appearance on their own they dissolve back to nonorigination.

This Trinity is THE most important philosophical point of all. And I cannot take credit for it. Evertying I know about it or have said here is based on what I’ve learned from Buddhism and quantum mechanics. In particular there are thousands of years of highly developed Buddhist logical treatises on precisely this point.

What is Actually Happening

When things happen they don’t just appear out of nothingness.

There isn’t really any nothingness. Nothingness is impossible by virtue of the following proof: Something appears right now. Nothing and something are mutually exclusive.

Furthemore, even IF nothing was possible, it could never generate anything because there is no way to turn nothingness into something other than nothingness.

Instead of nothingness there is a kind of space of knowing or being — what might be called awareness. This space is not inherently personalized — it has no concepts or sense of I or of being an observer, etc. This awareness has the characteristic of being nonoriginated — we cannot find it or call it a concrete, truly-existing, isolated “thing.”

At the same time as there is any knowing or being, appearances spontaneously develop within its scope. For example, this is just like dreaming. In a dream there is the space of the mind and then within this space various appearances (and other sensory experiences, for example of sound, etc.) unfold. We then identify with a particular character or perspective in the dream and the appearance of its body –
and we call that “I” or “self.” That is a habit — there is nothing inherently real about the character we see ourselves as in a dream — it is not really us, not really our body or our actual mind but rather just a dream image of a body and mind. We label it as “I” or “me” out of habit. In fact, our real body is alseep in bed and is not in the dream, and our real mind and self are having the dream they are not really in the dream. Or are they?

When we dream, dreams don’t appear out of nothing, they appear out of awareness.

The same goes for all the experiences (aka appearances in various sensory modalities) that we call a moment of “our universe.” At each moment of experience there is the space of awareness plus at least some appearance. Neither the awareness or the appearances are truly-existing or even separate, they are just two aspects of nonorigination.

Nonorigination — or what in Buddhism is called “emptiness” is not a final fundamental thing that can be grasped or found either — if you find it you find that it dissolves into awareness and appearances and these dissolve back into nonorigination, endlessly.

Time unfolds as the process of this infinite loop — the Trinity of nonorigination, awareness and appearance — iterating. We are always either looking at an appearance, our awareness, or nonorigination. In either case as soon as we make such an observation what we find is that these dissolve into their counterparts. As we keep observing we trigger the process of cause-and-effect which continues to perpetuate appearances and that is what powers the universe so to speak. The
energy we put into it by making observations drives it to “run” this program so to speak, endlessly iterating new moments of experience that then trigger us to make further observations and so on.

On a quantum level, the process of enacting awareness, via simple acts of observation — is literally what causes the universe to make quantum decisions that jolt the quantum field of possibilities to “collapse” onto a single possibility whenever we look for it. This is analogous to being able to cause liquid water to suddenly freeze into ice by just looking at it. When we don’t look, it’s water, but when we do look it instantly freezes into a particular shape.

We can never really see it in its water form, it always freezes just when we look for it. But we can infer the water from the frozen shapes that appear. Even ice has has waterlike qualities — it’s clear, and it melts back into water when heated after all. If we look closely at any observation (any shape made of ice in this analogy), to find its nature, this is analogous to heating the ice we are looking at, which melts it back to liquid form.

Once it melts we can no longer see it (in this analogy) until we make the next observation as we continue to look for it again. Our next observation is conditioned by the previous observation — the network of probabilities for what can appear next are changed by the previous observation — and this causes it to follow from it, statistically, rather than to be completely random — this is the process of cause-and-effect in a nutshell. Therefore our acts of observation crystallize and perpetuate our experience in an ongoing, recursive process.

Each act of observation effectively loads the dice for the next act of observation and so changes the odds of the next possible dicerolls. If the world did not work this way it would be totally random. Since it’s not totally random — it does appear to behave in a non-random fashion, we are able to make various kinds of predictions, there is a certain amount of consistency over time, this is how the universe must
and does work. Cause-and-effect makes the universe non-random and non-randomness of the universe results in cause-and-effect operating.

Metascience: What are the Possible Beliefs We Might Hold?

So far we have explored some very deep questions about the origin and nature of the universe and, if one believes in God, then of God too. We have found that all these questions converge on the same ultimate reality — the reality of nonorigination.

But while they may all converge on that point eventually, there are many different schools of thought within science and religion, and regarding how they relate to one another. So how do we choose what to believe in?

It is necessary to make such choices in order to simply function on a day-to-day level, to resolve difficult moral questions, and to figure out how to live or what to do in the future. Many people just accept the choice that is handed to them by their parents, or by authorities they trust. But if one has the freedom and presence of mind to question this themselves, then on what basis can an intelligent choice be made?

It’s difficult to make sense of the range of belief system choices available, and their biggest differences or main points. One could proceed on an extensive voyage of exploration — surveying every field of science and religion over decades (what I did by default). But the whole task might be a lot faster and more efficient if one had a map to start with.

I propose a field of thinking about what to believe that we might call “Metascience” in which we make maps to help people navigate possible belief systems more intelligently. In this approach we address big philosophical questions from a higher level, starting by enumerating the space of possible beliefs we could hold about them — rather than by starting with a particular choice of belief. (Note: Another word for Metascience might simply be philosophy or metaphysics. But Philosophy and more specifically, metaphysics, have gotten totally lost, irrelevant, and non-objective. It’s time for a refresh.).

So, regarding the choice of beliefs about the relatoinship between God and the universe — Instead of immediately diving into the rathole of arguing the specifics of any one particular belief system or position on the issues, first let’s at least try try to agree on what the set of possible beliefs and positions is, and on a way to enumerate them as elegantly and usefully as possible. Is a universally agreeable
metascience possible? Can we come up with a way to enumerate all the possible belief systems about God and the universe that everyone can agree with?

A Categorization of All Possible Beliefs About The Universe and God

So here is my first attempt at mapping out the exhaustive metascientific enumeration of all possible philosophies regarding God and the Universe.

(A) Hierarchical Approach: Either the universe or God is more fundamental and/or includes the other

Theories in which the universe – or all time and space – take place within God’s mind and/or body and is subject to God’s laws and will
Theories in which God exists as something within the universe, subject to it’s physical laws and conditions

(B) Dualistic Approach: The universe and God are two separate things

Theories in which God is the first-cause, creator or “blind watchmaker” who started the universe and then detached from it
Theories in which God is watching the universe from some place outside and separate from it and may or may not intervene
Theories in which God and the universe are separate things that co-exist within an even higher-order universe and/or pantheon
Theories in which either God or the universe is more potent or real than the other, and they are separate things

Theories in which the universe is a vast, intelligent, aware, sentient being of some sort (that we name “God”)
Theories in which God is just a synonym or label for the universe, or vice-versa.
The universe and God are a dichotomy; they are neither the same nor different. The universe and God are distinct but connected or merged together as one entity (e.g. God or the universe is considered to be the fundamental aspect and the other is considered to be relative aspect of the same dichotomy, the wave-particle duality, space-time, matter-energy, mind-body, one-many, etc.). Or in other words, theories in which God and the universe are two sides of the same coin so to speak — two distinct sides of the same thing
Theories in which either God exists and the universe doesn’t, or the universe exists and God doesn’t

(D) Existential Approach: The universe and/or God is a provisionally existing thing

Theories in which God or the universe has only a provisional kind of existence that when analyzed proves to reduce to a deeper level of existence, or to non-existence.
Theories in which God or the universe is merely a conceptual construct or label for something that actually has no valid existence of its own (e.g. “the horns of a rabbit”)
Theories in which God is a conceptual label for something that is impossible (e.g. “this statement is not true)
Theories in which God is a fictional character in a story (e.g. the character,
“Aslan” in the Chronicles of Narnia), or is a mental fabrication or projection of someone’s mind
Theories in which the universe is fictional but taking place – a mere fantasy or dream or a mental fabrication or projection of someone’s mind — it doesn’t exist in reality, it only exists in each of our own perceptions or at least in someone’s mind.
Theories such as nihilism which posit that there is actually nothing at all (a contradictory, and irrational assertion)

(E) Nonconceptual Approach: The universe and/or God is inconceivable

Theories in which the universe and/or God is neither posited to exist, not-exist, both exist and not-exist, or neither exist nor not-exist (e.g. the Buddhist theory of “emptiness” or “freedom from four logical extreme views”)
Theories in which God or the concept of the universe is a conceptualization of something real but inconceivable (e.g. “infinity” or “zero”)
Theories in which God and/or the universe cannot be conceived of for some axiomatic reason, such as being transcendental, beyond the scope of thought or words, beyond logic, not in the material realm, higher-dimensional, beyond time and space, etc.

There are no other major categories that I can think of regarding the Universe and God. I believe this may be then an exhaustive list. But feel free to add your own thoughts in the comments below.

Are These Questions Worthwhile?

At this point, for the skeptics among us, we should ask whether it is even meaningful and worthwhile to try to unify science and religion.

It is certainly clear that science has value. But what about religion?

Firstly, much of the world’s population believes in some form of religion and these beliefs are at the root of much of what takes place in the world — culturally, politically, economically and more. For that reason, if nothing else, we really should have as deep an understanding of all the various conceptions about God as we can. But that’s just the start. In fact there are sound scientific and philosophical reasons for exploring the topic of God as well. The theory that God originated the universe is just a valid a hypothesis as any other theory — and may even be testable at some point in the future. It’s certainly no more outlandish than some of the more exotic
and hard-to-test cosmological hypotheses put forth in recent decades.

In addition, many people (including even many scientists) have had personal experiences that indicate that there is some greater entity beyond the body, mind or individual self, and perhaps even beyond the physical limits of space and time. While not everyone has had such experiences, and there is no way to validate the experiences of others, the fact that such experiences are so common and so similar, is another data-point that makes this topic worthy of consideration both by those
who claim to have had such experiences, and by those who claim to have not had them. They may be artifacts of the particular architecture of the human body and brain, or they may be pointing to a deeper reality that exists just as objectively as the physical world.

Finally, from a purely scientific perspective, the origin of the universe is a mystery, and therefore the possibility of God is as much an open question as it ever was. Science has been able to learn about how the universe works to some degree, and to map parts of it, and even to form conjectures about how it has developed — but where it comes from, how it started (if it even has a beginning at all), and even
where it is located ultimately are a mystery. If one posits any kind of a beginning — such as a Big Bang — then that immediately begs the question of where did the Beginning come from?

Religion has certainly learned a lot from science over the millennia. But perhaps, ironically, science has as much to learn from religion in coming millennia, at least when it comes to understanding and exploring the farthest possible reaches of cosmology and the mind. The strange relationship between mind and matter may be what the next great scientific revolution will focus on.

Similarities Between Sciences and Religions

While science and religion may disagree on certain points, at the very deepest level, they may actually be more compatible than we might think. In fact, I would go so far as to propose that a grand unification of science and religion may come about in the future as we probe ever deeper into the edges of what we know about cosmology, subatomic physics, and even our understanding of consciousness and the mind.

The strangeness at the boundaries of science already points to a reality that goes beyond a strict division of mind and matter. For example, the simple act of observation seems to have an influence on what is actually measured to take place, according to the field of quantum mechanics. Similarly, at the borders of cosmology, questions still abound on the origin, structure, and fate of the universe. And in particular, given the improbabilty of a universe such as ours, which seems to be precisely balanced to support the emergence of intelligent life, how did this universe happen?

In many cases scientists are very careful to state that they simply don’t know certain things yet. But at the same time, as scienfitic theories come into vogue, they often get out of control. For example the theory of the Big Bang. This particular theory, like most other scientific theories, has gone from being a new and contentious proposal, to a major and mainstream scientific belief, to a term that
even non-scientists embraced as fact, and now today there is new evidence that perhaps the Big Bang theory is flawed and/or totally incorrect.

In the field of the philosophy of science, which studies how scientific paradigms are born, how they develop and compete, and how they are overturned, there are many other examples (the view of the Newtonian universe versus the view of Relativity, for example, or various explanations for the quantum world, and more recently String Theory). As scientific belief systems emerge, their proponents sometimes develop a kind of faith in the veracity of their beliefs that is not yet justified by the evidence, or that can never be justified in some cases — this scientific faith is quite similar to religious faith. It’s a strong belief in an explanation of nature for which there is some evidence but not yet final proof.

In fact, in science, theories can only be falsified, they can never be established as permanent and final. One never knows if and when new evidence may emerge that overturns the received view, or points to a deeper understanding.

It should also be noted that it is not the case that science is rational and religion is not. In fact, most if not all religions claim that that at least some of their beliefs are verifiable by individuals who follow a rational and repeatable process (for example, do certain things and you will get certain results). In addition at least some religions also apply rigorous formal logic to support their viewpoints. Those religions that provide an experimental method (do certain things and anyone will get predictable results) and that also apply rigorous logic to their reasoning, are applying a form of scientific method. It may be a weak form of scientific method, but it is not irrational.

So while science and religion have very different methodologies, at least with regard to their answers to the really Big Questions, such as the origin and ultimate nature of the universe, they both require a certain amount of faith, and they are both rational processes to some degree.

Differences Between Sciences and Religions

However there are also certain key differences between sciences and religions. In particular, many religions are built from axioms (creation myths, dieties, stories, traditions, and rules) which are established tautologically (they are considered to be true because simply they are defined to be true). For example, those religions which found their belief systems on ancient manuscripts that are said to have come directly for God, are building their belief systems from axioms. Such texts are claimed to be axiomatically true and cannot be disputed for they are God’s Word.

Some relgions also make the claim that the only way to test and verify the truth of their beliefs is to first take them on faith as true. In other words, the only way to verify that x is true is to first believe that x is true, and then after you believe it, the evidence will start to emerge. In other words, not having faith — asking questions or having doubts — actually prevents one from discovering the truth. It is the act of having faith that actually opens the door, so to speak.

Putting faith first is the opposite of the scientific method. The scientific method starts with doubt. It invites questioning — nothing is too sacred to examine, and if some theory can’t stand up to scrutiny, or can’t be shown through experiment or logic to be true, then it can’t be said to be scientific fact. In fact, to accept that
something is true without having doubts, but prior to having proof, would be a grave scientific error. This is a key difference between the methodologies of sciences and religions in general.

However, different though it may be from the scientific method, the religious approach seems to work. Billions of people throughout human history who have followed various religions have been able to verify, for themselves at least, the authenticity of their beliefs. Whether or not the stories in a certain religious text are literally true or only metaphorical or allegorical, the fact remains that the religious
process of faith, devotion, prayer and personal growth do lead, in a predictable and repeatable manner, to profound religious experiences and in some cases even to unexplainable “miracles” at times (such as the many documented cases of spontaneous healings, for example). While this is certainly not the scientific method, it appears to work pretty well nonetheless.

It is not my intention to prove that the scientific method of “proof before faith” is better or worse than the religious approach of “faith before proof.” In fact, I think they both have their place, and they both work, for different purposes.

The Boundary Between Science and Religion is Fuzzier Than One Might Think

The boundary between where science ends and religion begins is fuzzy at best. In fact, they are so intimately connected at the deepest levels that perhaps they will oneday turn out to be the same thing.

Already we have found that on the quantum scale there is an intimate and strange connection between conscious observation and what appears to happen. This is not well understood yet, but it is observed experimentally. Yet we don’t have any real understanding of what consciousness is, or how it interacts with what is observed. The sciences have very little understanding of the mind at all. In fact, many scientists don’t even believe there is a mind; they think the brain is a machine and the mind is a kind of illusion. There is no soul, no consciousness, no being at all. Yet others disagree. The jury is still out.

Religions on the other hand have been studying consciousness for millennia, and some are downright scientific about it. For example the ancient Hindu and Buddhist tantric sciences provide extremely detailed and sophisticated technologies for using the breath, posture, visualization, sound, and concentration to bring about extremely unusual states of body and mind (which have recently have been measured in scientific laboratories in a number of studies). Religions are in some ways way ahead of science when it comes to understanding the mind.

The mind is one of the places where science and religion are going to collide and most likely converge. Another is the ultimate nature of the universe — the nature of space and time. The boundary between science and religion becomes fuzzier as one begins to explore the mind, the relationship between mind and matter, and simply as one views the universe at the largest or smallest scales.

There have been many past attempts by scientists at proving and disproving the existence of God. In fact the question of God’s existence was once considered an acceptable topic of enquiry by scientists such as for example, Sir Isaac Newton, and many others. In the past science was concerned with all questions about nature –
including questions about the nature of reality and the mind, and even the possibility of a soul. But in recent times the focus of mainstream science has shifted far away from such topics — which are now seen as almost taboo. But why should they be taboo? They are just as much a subject for enquiry as ever. God has not been proved to exist or not-exist by science, and therefore the jury is still out. The question is whether there is any way to prove that God exists or not? It may in fact be possible to do this, scientifically, eventually.

In any case, just as is the case for the question of God, there are many scientific questions that also have not been answered yet, especially in the fields of cosmology and theoretical physics. Where does the universe come from? What created it? What came before the Big Bang (if there was a Big Bang)? What medium is space-time taking place in right now, or if there is nothing beyond space time then how did it ever happen, what does it come from, how could there be nothing beyond it? Does the universe have any edges and if so what is outside them? If
there are multiple universes, what separates them from each other, or are they connected and if so how? Do all possible states of all possible universes already exist or are they truly unfolding over time? Is everything predetermined by the physical laws, or is it all open to chance, or is there some level of intelligence and creativity taking place in the universe?

Even if science someday were able to describe and define everything there is to know about the physical universe, there would still be something more to know that could not be proved or discribed or defined. Godel’s famous Incompleteness proof established this on a formal logical level — there will always be gaps in our knowledge — of any formal systems we construct. No formal system can be both
consistent and complete at the same time. We will never have perfect scientific knowledge of the universe. And even if we could, it would simply beg the question of what is beyond that — no matter what we say the universe is, the question will always come up: well, then where does it come from and how or why is it happening?

Whether through science or religion, all paths lead to the possibility of something inconcievably beyond what we know. And this is where the boundary between science and religions gets so fuzzy that it dissolves completely.

Making a Choice

Assuming we can all at least agree on the meta-level choices (the set of possible choices), we can then discuss possible criteria for comparing, testing, and even ranking the various possible choices available to us.

At the end of the process of course there may be no final best choice that everyone accepts (in fact, I can guarantee there will not be!), nor any agreement as to what are the best or correct criteria for choosing among them. But at least we can all at least agree on what the choices are and how they compare to one another in various ways.

This could go a long way to promoting and improving tolerance and understanding. Better yet, this kind of process might even lead to useful meta-level or inter-belief-system dialogues that may eventually lead to important discoveries and even grand unifications in the future.

However, for now, regardless of what belief system we prefer, we simply have to accept that the belief system we choose, if any, is a matter of personal choice (some might call that faith, others might call it aesthetic preference, others might call it a hunch or intuition) — at least until such time as someone comes up with a way to objectively prove to everyone else that there is only one correct choice. Until that time, even if we have our own favorite belief system choice, we still have to keep some measure of open-mindedness in the face of the set of other choices available and the fact that we can’t today prove objectively (to everyone) that we made the right choice.

At least however, we should be clear that if we are willing to believe anything about the universe, there are strong reasons why we therefore should keep an open mind with regard to the possibility of God. It is not that huge a leap in fact. If we are willing to accept that something as vast and inconceivable as the universe exists, then why not God too? We really don’t have much solid grounds for holding any beliefs about such things — to do so is really just an act of faith either way. We should not have illusions about that. Believing in scientific explanations of the cosmos is really not that much different than believing in religous ones.

The good news at least is that so long as our conception of God has the properties of being uncaused and unconditioned, we are likely to have made the right choice. This also means that all the great religions, at least at their cores, are in agreement — they are all worshipping the same ultimate God, regardless of what different names they use for it. You really can’t go wrong as long as you believe in an ultimate nature that is uncaused and uncreated. However — where you certainly CAN go wrong is in imposing any further beliefs on it. And many make that mistake.

Nonduality

I have shown in this article that if one believes in the physical universe described by science, then in fact there is a logical requirement that the universe is ultimately nonoriginated.

I have also shown that the same holds for belief in God — God is also logically required to be nonoriginated.

Therefore the universe and God have the same ultimate nature.

In addition I have shown that for the universe to make choices about what happens from the set of all possibilities, observation, and therefore awareness, is required. Furthermore the nature of sentient beings, and of God, is precisely this unique capacity of awareness. Both the universe and what we think of as God are characterized by the same nature of being nonoriginated and aware.

In fact, at this level, the ultimate nature is not very different from the core idea of what God is. On an ultimate level there is not really much of a distinction between the ultimate nature of the universe and the ultimate nature of God — it is just one ultimate reality. The universe and God may be one thing, or they may be two
things, or only one and not the other may exist, but in any and all of these cases, there is still only one ultimate nature: nonoriginated awareness.

There is no escape from this logic. There is no question that somewhere down the line, we must finally accept that there is something greater than the universe — whatever we think the universe is — and the characterstics of that greater thing are in fact the one common theme of the conception of God across all religions. We can name it what we want, and certainly different religions do. We also may have different perspectives on it, and add all sorts of other details. But what all the great religions have in common is an ultimate nature that is essentially transcendental.

In other words, science and religion are two sides of the same coin. You really can’t have one without the other. They are a dichotomy, but not a duality. They are distinct yet unified.

We do however have the freedom to choose our relative level beliefs about science, and our religious tradition. This freedom is an expression of the primordial freedom of the awareness — our ability to choose what to observe — and this in turn is the ultimate nature of reality. Intellectual freedom is therefore not only irrepressible, it is a reflection of the nature of the universe, it is our birthright.

On the ultimate level everything is unified, but on the relative level, there is no one correct science or religion, there will always be different views, and they probably won’t always agree on all points, and this is perfectly in accord with the freedom of the universe, and each individual. So while science and religion may be unified on the ultimate level, they certainly are not unified on the relative level, and in fact even within each indivividual field of science and each religion, there are differing viewpoints and schools of thought. And this is good.

There is a menu of different belief systems in both arenas and various items on the menu are or are not compatible with one another, or with the beliefs of others. It’s really our personal choice to make. However, what should be clear from the above argument is we have to choose both a main course and a desert: science is undeniable, and religion is unavoidable, they are two sides of the same coin.

Science and religion are different on the relative level (though not as different as some might think), but they definitely converge at ultimate level and this convergence is not a matter of faith, it is a matter of logic. Therefore, regardless of whether we prefer science or religion, or any particular sect within either camp, at least we should not err on the side of thinking they are mutually exclusive.

Unifying Physics and Consciousness: The Next Scientific Revolution

If you pursue science to the very edges, you reach nonorigination. Similarly if you become as close as possible to the diety in any religious tradition, you reach nonorigination. Moreover, nonorigination is the nature of appearances and awareness, and vice-verse. They are never separated. It’s a trinity.

The ultimate nature of the universe, and the ultimate nature of God (if you believe in a God) – must logically be precisely the same. This nature unifies the physical world of seemingly “external” sensory experiences and seemingly “internal” mental events, with the unfindable yet undeniable dimension of awareness, and the unfindable yet logically required nature of being nonoriginated.

The beauty of this is that on the ultimate level there really is no question at all about whether or not the universe exists, or whether or not God exists — the appearances of primordially aware nonorigination is the truth — and it is the most amazing miracle of all. It is irrefutable, it is logically required, and it establishes a basis for authentic and universal spirituality. One can logically derive or directly experience this logical trinity through the vehicle of focusing on and logically analyzing any phenomena (the universe, the mind, God, etc.). When this trinity is recognized as the nature of reality, and directly experienced as such, that is the deepest scientific observation or religious experience possible.

The universe including the body and all other physical things in space and time, the conceptual mind and its mental realm of thoughts and emotions, and all possible real or imaginary dieties, all have at their ultimate root, the same primordially nonoriginated awareness.

Proving this once and for all in a non-religiously couched manner — using pure logical reasoning — enables science to progress beyond its present day limitations to finally begin to make sense of the strangeness of the quantum world and of the role and nature of consciousness, and the ultimate nature of space and time.

The next frontier in science will not be simply be a deeper understanding of the physical world — it will be a broader and more integrated understanding that includes both the physical world and the realm of consciousness — the mental realm.

To fully explain and understand the physical world science must find ways to include and measure the crucial role of conscious observers. Each physical event has both sides on a quantum level: the side of the observer and the side of what is observed. Science has so far been focused exclusively on understanding the side of what is observed. But what is observed cannot fully be understood or explained without an equal measure of scientific understanding of the observer and the act of observation.

Similarly, the only way to fully understand consciousness is to include and measure the crucial relationship between consciousness and the process of appearance (namely cause and effect). Both the physical world and consciousness are nonoriginated — they are empty of having an origin, not having an origin, having both, or having neither.

We don’t have the tools for measuring or exploring consciousness yet, but we’re close. Experiments that show the impact of observation on reality are indicators that consciousness is a phenomenon that can affect the observable world. This means that consciousness is indirectly detectable via measurments of the physical world around observers. It may be that consciousness — the act of observing — cannot be directly measured or observed except on its own — by and “within” each individual — but may still me indirectly measured or detected via its affects on the quantum field in the environment when it is present.

By analogy, this is similar to how space is measured, so it is possible to imagine doing this for consciousness. In the case of space, we cannot see it, touch it, or measure it directly. We can only infer things about it by measuring other things — like the way light travels, or the way things move. These indirect measurements lead us to an understanding of space.

Similarly we may be able to triangulate on consciousness by measuring the effects of various physical changes on consciousness (as reported by a conscious observer) and/or by the effects of consciousness (some observer) on physical phenomena (such as the Double Slit experiment). This is definitely an interesting possibility for
further exploration, and perhaps the next scientific revolution is waiting just over the horizon in this direction.

Our civilization has not even scratched the surface of this new frontier — a unified science of physics and consciousness. But we will soon. We have to. It is unavoidable. Our quest for knowledge and understanding will take us there whether we like it or not. Already there are cracks in our present scientific theories, and experiments are showing us gaps and contradictions in our theories that we cannot explain. And the light is spilling through them.

Video: My Talk on the Evolution of the Global Brain at the Singularity Summit

Nova — Sat, 14 Feb 2009 07:32:41 +0000

If you are interested in collective intelligence, consciousness, the global brain and the evolution of artificial intelligence and superhuman intelligence, you may want to see my talk at the 2008 Singularity Summit. The videos from the Summit have just come online.

(Many thanks to Hrafn Thorisson who worked with me as my research assistant for this talk).

Watch My best Talk: The Global Brain is Coming

Nova — Thu, 02 Oct 2008 19:56:57 +0000

I’ve posted a link to a video of my best talk — given at the GRID ‘08 Conference in Stockholm this summer. It’s about the growth of collective intelligence and the Semantic Web, and the future and role the media. Read more and get the video here. Enjoy!

Life in Perpetual Beta: The Film

Nova — Wed, 11 Jun 2008 14:41:40 +0000

Melissa Pierce is a filmmaker who is making a film about "Life in Perpetual Beta." It’s about how people who are adapting and reinventing themselves in the moment, and a new philosophy or approach to life. She’s interviewed a number of interesting people, and while I was in Chicago recently, she spoke with me as well. Here is a clip about how I view the philosophy of living in Beta. Her film is also in perpetual beta, and you can see the clips from her interviews on her blog as the film evolves. Eventually it will be released through the indie film circuit, and it looks like it will be a cool film. By the way, she is open to getting sponsors so if you like this idea and want your brand on the opening credits, drop her a line!

A Universal Classification of Intelligence

Nova — Tue, 05 Feb 2008 18:21:06 +0000

I’ve been thinking lately about whether or not it is possible to formulate a scale of universal cognitive capabilities, such that any intelligent system — whether naturally occurring or synthetic — can be classified according to its cognitive capacity. Such a system would provide us with a normalized scientific basis by which to quantify and compare the relative cognitive capabilities of artificially intelligent systems, various species of intelligent life on Earth, and perhaps even intelligent lifeforms encountered on other planets.

One approach to such evaluation is to use a standardized test, such as an IQ test. However, this test is far too primitive and biased towards human intelligence. A dolphin would do poorly on our standardized IQ test, but that doesn’t mean much, because the test itself is geared towards humans. What is needed is a way to evaluate and compare intelligence across different species — one that is much more granular and basic.

What we need is a system that focuses on basic building blocks of intelligence, starting by measuring the presence or ability to work with fundamental cognitive constructs (such as the notion of object constancy, quantities, basic arithmetic constructs, self-constructs, etc.) and moving up towards higher-level abstractions and procedural capabilities (self-awareness, time, space, spatial and temporal reasoning, metaphors, sets, language, induction, logical reasoning, etc.).

What I am asking is whether we can develop a more "universal" way to rate and compare intelligences? Such a system would provide a way to formally evaluate and rate any kind of intelligent system — whether insect, animal, human, software, or alien — in a normalized manner.

Beyond the inherent utility of having such a rating scale, there is an additional benefit to trying to formulate this system: It will lead us to really question and explore the nature of cognition itself. I believe we are moving into an age of intelligence — an age where humanity will explore the brain and the mind (the true "final frontier"). In order to explore this frontier, we need a map — and the rating scale I am calling for would provide us with one, for it maps the range of possible capabilities that intelligent systems are capable of.

I’m not as concerned with measuring the degree to which any system is more or less capable of some particular cognitive capability within the space of possible capabilities we map (such as how fast it can do algebra for example, or how well it can recall memories, etc.) — but that is a useful second step. The first step, however, is to simply provide a comprehensive map of all the possible fundamental cognitive behaviors there are — and to make this map as minimal and elegant as we can. Ideally we should be seeking the simplest set of cognitive building blocks from which all cognitive behavior, and therefore all minds, are comprised.

So the question is: Are there in fact "cognitive universals" or universal cognitive capabilities that we can generalize across all possible intelligent systems? This is a fascinating question — although we are human, can we not only imagine, but even prove, that there is a set of basic universal cognitive capabilities that applies everywhere in the universe, or even in other possible universes? This is an exploration that leads into the region where science, pure math, philosophy, and perhaps even spirituality all converge. Ultimately, this map must cover the full range of cognitive capabilities from the most mundane, to what might be (from our perspective) paranormal, or even in the realm of science fiction. Ordinary cognition as well as forms of altered or unhealthy cognition, as well as highly advanced or even what might be said to be enlightened cognition, all have to fit into this model.

Can we develop a system that would apply not just to any form of intelligence on Earth, but even to far-flung intelligent organisms that might exist on other worlds, and that perhaps might exist in dramatically different environments than humans? And how might we develop and test this model?

I would propose that such a system could be developed and tuned by testing it across the range of forms of intelligent life we find on Earth — including social insects (termite colonies, bee hives, etc.), a wide range of other animal species (dogs, birds, chimpanzees, dolphins, whales, etc.), human individuals, and human social organizations (teams, communities, enterprises). Since there are very few examples of artificial intelligence today it would be hard to find suitable systems to test it on, but perhaps there may be a few candidates in the next decade. We should also attempt to imagine forms of intelligence on other planets that might have extremely different sensory capabilities, totally different bodies, and perhaps that exist on very different timescales or spatial scales as well — what would such exotic, alien intelligences be like, and can our model encompass the basic building blocks of their cognition as well?

It will take decades to develop and tune a system such as this, and as we learn more about the brain and the mind, we will continue to add subtlety to the model. But when humanity finally establishes open dialog with an extraterrestrial civilization, perhaps via SETI or some other means of more direct contact, we will reap important rewards. A system such as what I am proposing will provide us with a valuable map for understanding alien cognition, and that may prove to be the key to enabling humanity to engage in successful interactions and relations with alien civilizations as we may inevitably encounter as humanity spreads throughout the galaxy. While some skeptics may claim that we will never encounter intelligent life on other planets, the odds would indicate otherwise. It may take a long time, but eventually it is inevitable that we will cross paths — if they exist at all. Not to be prepared would be irresponsible.

Artificial Stupidity: The Next Big Thing

Nova — Thu, 24 Jan 2008 21:13:45 +0000

There has been a lot of hype about artificial intelligence over the years. And recently it seems there has been a resurgence in interest in this topic in the media. But artificial intelligence scares me. And frankly, I don’t need it. My human intelligence is quite good, thank you very much. And as far as trusting computers to make intelligent decisions on my behalf, I’m skeptical to say the least. I don’t need or want artificial intelligence.

No, what I really need is artificial stupidity.

I need software that will automate all the stupid things I presently have to waste far too much of my valuable time on. I need something to do all the stupid tasks — like organizing email, filing documents, organizing folders, remembering things, coordinating schedules, finding things that are of interest, filtering out things that are not of interest, responding to routine messages, re-organizing things, linking things, tracking things, researching prices and deals, and the many other rote information tasks I deal with every day.

The human brain is the result of millions of years of evolution. It’s already the most intelligent thing on this planet. Why are we wasting so much of our brainpower on tasks that don’t require intelligence? The next revolution in software and the Web is not going to be artificial intelligence, it’s going to be creating artificial stupidity: systems that can do a really good job at the stupid stuff, so we have more time to use our intelligence for higher level thinking.

The next wave of software and the Web will be about making software and the Web smarter. But when we say "smarter" we don’t mean smart like a human is smart, we mean "smarter at doing the stupid things that humans aren’t good at." In fact humans are really bad at doing relatively simple, "stupid" things — tasks that don’t require much intelligence at all.

For example, organizing. We are terrible organizers. We are lazy, messy, inconsistent, and we make all kinds of errors by accident. We are terrible at tagging and linking as well, it turns out. We are terrible at coordinating or tracking multiple things at once because we are easily overloaded and we can really only do one thing well at a time. These kinds of tasks are just not what our brains are good at. That’s what computers are for – or should be for at least.

Humans are really good at higher level cognition: complex thinking, decisionmaking, learning, teaching, inventing, expressing, exploring, planning, reasoning, sensemaking, and problem solving — but we are just terrible at managing email, or making sense of the Web. Let’s play to our strengths and use computers to compensate for our weaknesses.

I think it’s time we stop talking about artificial intelligence — which nobody really needs, and fewer will ever trust. Instead we should be working on artificial stupidity. Sometimes the less lofty goals are the ones that turn out to be most useful in the end.

Virtual Out of Body Experiences

Nova — Sat, 25 Aug 2007 14:45:10 +0000

A very cool experiment in virtual reality has shown it is possible to trick the mind into identifying with a virtual body:

Through these goggles, the volunteers could see a camera
view of their own back – a three-dimensional "virtual own body" that
appeared to be standing in front of them.

When the researchers stroked the back of the volunteer
with a pen, the volunteer could see their virtual back being stroked
either simultaneously or with a time lag.

The volunteers reported that the sensation seemed to be
caused by the pen on their virtual back, rather than their real back,
making them feel as if the virtual body was their own rather than a
hologram.

Volunteers

Even when the camera was switched to film the back of a
mannequin being stroked rather than their own back, the volunteers
still reported feeling as if the virtual mannequin body was their own.

And when the researchers switched off the goggles,
guided the volunteers back a few paces, and then asked them to walk
back to where they had been standing, the volunteers overshot the
target, returning nearer to the position of their "virtual self".

This has implications for next-generation video games and virtual reality. It also has interesting implications for consciousness studies in general.

ool

Enriching the Connections of the Web — Making the Web Smarter

Nova — Tue, 03 Jul 2007 20:27:18 +0000

Web 3.0 — aka The Semantic Web — is about enriching the connections of the Web. By enriching the connections within the Web, the entire Web may become smarter.

I believe that collective intelligence primarily comes from connections — this is certainly the case in the brain where the number of connections between neurons far outnumbers the number of neurons; certainly there is more "intelligence" encoded in the brain’s connections than in the neurons alone. There are several kinds of connections on the Web:

Connections between information (such as links)
Connections between people (such as opt-in social relationships, buddy lists, etc.)
Connections between applications (web services, mashups, client server sessions, etc.)
Connections between information and people (personal data collections, blogs, social bookmarking, search results, etc.)
Connections between information and applications (databases and data sets stored or accessible by particular apps)
Connections between people and applications (user accounts, preferences, cookies, etc.)

Are there other kinds of connections that I haven’t listed — please let me know!

I believe that the Semantic Web can actually enrich all of these types of connections, adding more semantics not only to the things being connected (such as representations of information or people or apps) but also to the connections themselves.

In the Semantic Web approach, connections are represented with statements of the form (subject, predicate, object) where the elements have URIs that connect them to various ontologies where their precise intended meaning can be defined. These simple statements are sometimes called "triples" because they have three elements. In fact, many of us are working with statements that have more than three elements ("tuples"), so that we can represent not only subject, predicate, object of statements, but also things like provenance (where did the data for the statement come from?), timestamp (when was the statement made), and other attributes. There really is no limit to what kind of metadata can be stored in these statements. It’s a very simple, yet very flexible and extensible data model that can represent any kind of data structure.

The important point for this article however is that in this data model rather than there being just a single type of connection (as is the case on the present Web which basically just provides the HREF hotlink, which simply means "A and B are linked" and may carry minimal metadata in some cases), the Semantic Web enables an infinite range of arbitrarily defined connections to be used. The meaning of these connections can be very specific or very general.

For example one might define a type of connection called "friend of" or a type of connection called "employee of" — these have very different meanings (different semantics) which can be made explicit and also machine-readable using OWL. By linking a page about a person with the "employee of" link to another page about a different person, we can express that one of them employs the other. That is a statement that any application which can read OWL is able to see and correctly interpret, by referencing the underlying definition of "employee of" which is defined in some ontology and might for example specify that an "employee of" relation connects a person to a person or organization who is their employer. In other words, rather than just linking things with the generic "hotlink" we are all used to, they can now be linked with specific kinds of links that have very particular and unambiguous meaning and logical implications.

This has the potential at least to dramatically enrich the information-carrying capacity of connections (links) on the Web. It means that connections can carry more meaning, on their own. It’s a new place to put meaning in fact — you can put meaning between things to express their relationships. And since connections (links) far outnumber objects (information, people or applications) on the Web, this means we can radically improve the semantics of the structure of the Web as a whole — the Web can become more meaningful, literally. This makes a difference, even if all we do is just enrich connections between gross-level objects (in other words, connections between Web pages or data records, as opposed to connections between concepts expressed within them, such as for example, people and companies mentioned within a single document).

Even if the granularity of this improvement in connection technology is relatively gross level it could still be a major improvement to the Web. The long-term implications of this have hardly been imagined let alone understood — it is analogous to upgrading the dendrites in the human brain; it could be a catalyst for new levels of computation and intelligence to emerge.

It is important to note that, as illustrated above, there are many types of connections that involve people. In other words the Semantic Web, and Web 3.0, are just as much about people as they are about other things. Rather than excluding people, they actually enrich their relationships to other things. The Semantic Web, should, among other things, enable dramatically better social networking and collaboration to take place on the Web. It is not only about enriching content.

Now where will all these rich semantic connections come from? That’s the billion dollar question. Personally I think they will come from many places: from end-users as they find things, author content, bookmark content, share content and comment on content (just as hotlinks come from people today), as well as from applications which mine the Web and automatically create them. Note that even when Mining the Web a lot of the data actually still comes from people — for example, mining the Wikipedia, or a social network yields lots of great data that was ultimately extracted from user-contributions. So mining and artificial intelligence does not always imply "replacing people" — far from it! In fact, mining is often best applied as a means to effectively leverage the collective intelligence of millions of people.

These are subtle points that are very hard for non-specialists to see — without actually working with the underlying technologies such as RDF and OWL they are basically impossible to see right now. But soon there will be a range of Semantically-powered end-user-facing apps that will demonstrate this quite obviously. Stay tuned!

Of course these are just my opinions from years of hands-on experience with this stuff, but you are free to disagree or add to what I’m saying. I think there is something big happening though. Upgrading the connections of the Web is bound to have a significant effect on how the Web functions. It may take a while for all this to unfold however. I think we need to think in decades about big changes of this nature.

Web 3.0 — Next-Step for Web?

Nova — Tue, 03 Jul 2007 15:28:36 +0000

The Business 2.0 Article on Radar Networks and the Semantic Web just came online. It’s a huge article. In many ways it’s one of the best popular articles written about the Semantic Web in the mainstream press. It also goes into a lot of detail about what Radar Networks is working on.

One point of clarification, just in case anyone is wondering…

Web 3.0 is not just about machines — it’s actually all about humans — it leverages social networks, folksonomies, communities and social filtering AS WELL AS the Semantic Web, data mining, and artificial intelligence. The combination of the two is more powerful than either one on it’s own. Web 3.0 is Web 2.0 + 1. It’s NOT Web 2.0 – people. The "+ 1" is the
addition of software and metadata that help people and other
applications organize and make better sense of the Web. That new layer
of semantics — often called "The Semantic Web" — will add to and
build on the existing value provided by social networks, folksonomies,
and collaborative filtering that are already on the Web.

So at least here at Radar Networks, we are focusing much of our effort on facilitating people to help them help themselves, and to help each other, make sense of the Web. We leverage the amazing intelligence of the human brain, and we augment that using the Semantic Web, data mining, and artificial intelligence. We really believe that the next generation of collective intelligence is about creating systems of experts not expert systems.

Breaking the Collective IQ Barrier — Making Groups Smarter

Nova — Sat, 03 Mar 2007 23:46:40 +0000

I’ve been thinking since 1994 about how to get past a fundamental barrier to human social progress, which I call “The Collective IQ Barrier.” Most recently I have been approaching this challenge in the products we are developing at my stealth venture, Radar Networks.

In a nutshell, here is how I define this barrier:

The Collective IQ Barrier: The potential collective intelligence of a human group is exponentially proportional to group size, however in practice the actual collective intelligence that is achieved by a group is inversely proportional to group size. There is a huge delta between potential collective intelligence and actual collective intelligence in practice. In other words, when it comes to collective intelligence, the whole has the potential to be smarter than the sum of its parts, but in practice it is usually dumber.

Why does this barrier exist? Why are groups generally so bad at tapping the full potential of their collective intelligence? Why is it that smaller groups are so much better than large groups at innovation, decision-making, learning, problem solving, implementing solutions, and harnessing collective knowledge and intelligence?

I think the problem is technological, not social, at its core. In this article I will discuss the problem in more depth and then I will discuss why I think the Semantic Web may be the critical enabling technology for breaking through the Collective IQ Barrier.

The Effective Size of Groups

For millions of years — in fact since the dawn of humanity — humansocial organizations have been limited in effective size. Groups aremost effective when they are small, but they have less collectiveknowledge at their disposal. Slightly larger groups optimize both effectiveness and access to resources such as knowledge and expertise. In my own experience working on many different kinds of teams, I think that the sweet-spot is between 20and 50 people. Above this size groups rapidly become inefficient andunproductive.

The Invention of Hierarchy

The solution that humans have used to get around this limitation in the effective size of groups is hierarchy.When organizations grow beyond 50 people we start to break them intosub-organizations of less than 50 people. As a result if you look atany large organization, such as a Fortune 100 corporation, you find ahuge complex hierarchy of nested organizations and cross-functionalorganizations. This hierarchy enables the organization to createspecialized “cells” or “organs” of collective cognition aroundparticular domains (like sales, marketing, engineering, HR, strategy,etc.) that remain effective despite the overall size of theorganization.

By leveraging hierarchy an organization of even hundreds ofthousands of members can still achieve some level of collective IQ as awhole. The problem however is that the collective IQ of the wholeorganization is still quite a bit lower than the combined collectiveIQ’s of the sub-organizations that comprise it. Even in well-structured, well-managed hierarchies, the hierarchy is still less thanthe sum of it’s parts. Hierarchy also has limits — the collective IQof an organization is also inversely proportional to the number ofgroups it contains, and the average number of levels of hierarchybetween those groups (Perhaps this could be defined more elegantly asan inverse function of the average network distance between groups inan organization).

The reason that organizations today still have to make suchextensive use of hierarchy is that our technologies for managingcollaboration, community, knowledge and intelligence on a collectivescale are still extremely primitive. Hierarchy is still one of the only and best solutions we have at our disposal. But we’re getting better fast.

Modern organizations are larger and far more complex than ever would have beenpractical in the Middle Ages, for example. They contain more people,distributed more widely around the globe, with more collaboration andspecialization, and more information, making more rapid decisions, thanwas possible even 100 years ago. This is progress.

Enabling Technologies

There have beenseveral key technologies that made modern organizations possible: the printing press,telegraph, telephone, automobile, airplane, typewriter, radio,television, fax machine, and personal computer. These technologies haveenabled information and materials to flow more rapidly, at less cost,across ever more widely distributed organizations. So we can see that technology does make a big difference in organizational productivity. The question is, can technology get us beyond the Collective IQ Barrier?

The advent of the Internet, and in particular the World Wide Webenabled a big leap forward in collective intelligence. These technologies havefurther reduced the cost to distributing and accessing information andinformation products (and even “machines” in the form of software codeand Web services). They have made it possible for collectiveintelligence to function more rapidly, more dynamically, on a wider scale, and at lesscost, than any previous generation of technology.

As a result of evolution of the Web we have seen new organizationalstructures begin to emerge that are less hierarchical, moredistributed, and often more fluid. For example, virtual teams that caninstantly form, collaborate across boundaries, and then dissolve backinto the Webs they come from when their job is finished. Thisprocess is now much easier than it ever was. Numerous hosted Web-basedtools exist to facilitate this: email, groupware, wikis, messageboards, listservers, weblogs, hosted databases, social networks, searchportals, enterprise portals, etc.

But this is still just the cusp of this trend. Even today with thecurrent generation of Web-based tools available to us, we are still notable to effectively tap much more of the potential Collective IQ of ourgroups, teams and communities. How do we get from where we are today(the whole is dumber than the sum of its parts) to where we want to bein the future (the whole is smarter than the sum of its parts)?

The Future of Productivity

The diagram below illustrates how I think about the past, present and future of productivity. In my view, from the advent of PC’s onwards we have seen a rapid growth in individual and group productivity, enabling people to work with larger sets of information, in larger groups. But this will not last — soon as we reach a critical level of information and groups of ever larger size, productivity will start to decline again, unless new technologies and tools emerge to enable us to cope with these increases in scale and complexity. You can read more about this diagram here.

http://novaspivack.typepad.com/nova_spivacks_weblog/2007/02/steps_towards_a.html

In the last 20 years the amount of information that knowledgeworkers (and even consumers) have to deal with on a daily basis has mushroomed by a factor of almost 10orders of magnitude and it will continue like this for several moredecades. But our information tools — and particular our tools forcommunication, collaboration, community, commerce and knowledgemanagement — have not advanced nearly as quickly. As a result thetools that we are using today to manage our information andinteractions are grossly inadequate for the task at hand: They were simply not designed tohandle the tremendous volumes of distributed information, and the rate of change ofinformation, that we are witnessing today.

Case in point: Email. Email was never designed for what it is beingused for today. Email was a simple interpersonal notification andmessaging tool and essentially that is what it is good for. But todaymost of us use our email as a kind of database, search engine,collaboration tool, knowledge management tool, project management tool,community tool, commerce tool, content distribution tool, etc. Emailwasn’t designed for these functions and it really isn’t very productive whenapplied to them.

For groups the email problem is even worse than it is for individuals –not only is everyone’s individual email productivity declining anyway,but collectively as groupsize increases (and thus group information size increases as well),there is a multiplier effect that further reduces everyone’semail productivity in inverse proportion to the size of the group.Email becomes increasingly unproductive as group size and informationsize increase.

This is not just true of email, however, it’s true of almost all theinformation tools we use today: Search engines, wikis, groupware,social networks, etc. They all suffer from this fundamental problem.Productivity breaks down with scale — and the problem is exponentially worse than it is for individuals in groups and organizations. But scale is increasing incessantly — that is a fact — and it will continue to do so for decades at least. Unless something is done about this we will simply be completely buried in our own information within about a decade.

The Semantic Web

I think the Semantic Web is a critical enabling technology that will help us get through this transition. It willenable the next big leap in productivity and collective intelligence.It may even be the technology that enables humans to flip the ratio so thatfor the first time in human history, larger groups of people canfunction more productively and intelligently than smaller groups. Itall comes down to enabling individuals and groups to maintain (andultimately improve) their productivity in theface of the continuing explosion in information and social complexitythat they areexperiencing.

The Semantic Web provides a richer underlying fabric for expressing,sharing, and connecting information. Essentially it provides a betterway to transform information into useful knowledge, and to share andcollaborate with it. It essentially upgrades the medium — in this case the Web and any other data that is connected to the Web — that we use for our information today.

By enriching the medium we can inturn enable new leaps in how applications, people, groups andorganizations can function. This has happened many times before in thehistory of technology. The printing press is one example. The Web is a more recent one. The Web enriched themedium (documents) with HTML and a new transport mechanism, HTTP, forsharing it. This brought about one of the largest leaps in humancollective cognition and productivity in history. But HTML really onlydescribes formatting and links. XML came next, to start to provide away to enrich the medium with information about structure –the parts of documents. The Semantic Web takes this one step further –it provides a way to enrich the medium with information about the meaning of the structure — what are those parts, what do various links actually mean?

Essentially the Semantic Web provides a means to abstract andexternalize human knowledge about information — previously the meaningof information lived only in our heads, and perhaps in certainspecially-written software applications that were coded to understandcertain types of data. The Semantic Web will disrupt this situation by providingopen-standards for encoding this meaning right into the medium itself.Any application that can speak the open-standards of the Semantic Webcan then begin to correctly interpret the meaning of information, andtreat it accordingly, without having to be specifically coded tounderstand each type of data it might encounter.

This is analogous to the benefit of HTML. Before HTML everyapplication had to be specifically coded to each different documentformat in order to display it. After HTML applications could all juststandardize on a single way to define the formats of differentdocuments. Suddenly a huge new landscape of information becameaccessible both to applications and to the people who used them.The Semantic Web does something similar: It provides a way to makethe data itself “smarter” so that applications don’t have to know somuch to correctly interpret it. Any data structure — a document or adata record of any kind — that can be marked up with HTML to define its formatting, can also be marked up with RDFand OWL (the languages of the Semantic Web) to define its meaning.

Once semantic metadata is added, the document can not only bedisplayed properly by any application (thanks to HTML and XML), but itcan also be correctly understood by that application. For example theapplication can understand what kind of document it is, what it isabout, what the parts are, how the document relates to other things,and what particular data fields and values mean and how they map todata fields and values in other data records around the Web.

The Semantic Web enriches information with knowledge about what thatinformation means, what it is for, and how it relates to other things.With this in hand applications can go far beyond the limitations ofkeyword search, text processing, and brittle tabular data structures.Applications can start to do a much better job of finding, organizing,filtering, integrating, and making sense of ever larger and morecomplex distributed data sets around the Web.

Another great benefit ofthe Semantic Web is that this additional metadata can be added in atotally distributed fashion. The publisher of a document can add theirown metadata and other parties can then annotate that with their ownmetadata. Even HTML doesn’t enable that level of cooperative markup (exceptperhaps in wikis). It takes a distributed solution to keep up with ahighly distributed problem (the Web). The Semantic Web is just such adistributed solution.

The Semantic Web will enrich information and this in turn will enable people, groups and applications to work with information more productively. In particular groups and organizations will benefit the most because that is where the problems of information overload and complexity are the worst. Individuals at least know how they organize their own information so they can do a reasonably good job of managing their own data. But groups are another story — because people don’t necessarily know how others in their group organize their information. Finding what you need in other people’s information is much harder than finding it in your own.

Where the Semantic Web can help with this is by providing a richer fabric for knowledge management. Information can be connected to an underlying ontology that defines not only the types of information available, but also the meaning and relationships between different tags or subject categories, and even the concepts that occur in the information itself. This makes organizing and finding group knowledge easier. In fact, eventually the hope is that people and groups will not have to organize their information manually anymore — it will happen in an almost fully-automatic fashion. The Semantic Web provides the necessary frameworks for making this possible.

But even with the Semantic Web in place and widely adopted, moreinnovation on top of it will be necessary before we can truly breakpast the Collective IQ Barrier such that organizations can in practiceachieve exponential increases in Collective IQ. Human beings are only able to cope with a few chunks ofinformation at a given moment, and our memories and ability to processcomplex data sets are limited. When group size and data size growbeyond certain limits, we simply cannot cope, we become overloaded andjammed, even with rich Semantic Web content at our disposal.

Social Filtering and Social Networking — Collective Cognition

Ultimately, to remain productive in the face of such complexity wewill need help. Often humans in roles that require them to cope with large scales of information, relationships andcomplexity hire assistants, but not all of us can affordto do that, and in some cases even assistants are not able to keep upwith the complexity that has to be managed.

Social networking andsocial filtering are two ways to expand the number of “assistants” weeach have access to, while also reducing the price of harnessing the collective intelligence of those assistants to just about nothing. Essentially these methodologies enable people toleverage the combined intelligence and attention of large communitiesof like-minded people who contribute their knowledge and expertise for free. It’s a collective tit-for-tat form of altruism.

For example, Diggis a community that discovers the most interesting news articles. Itdoes this by enabling thousands of people to submit articles and voteon them. What Digg adds are a few clever algorithms on top of this for rankingarticles such that the most active ones bubble up to the top. It’s notunlike a stock market trader’s terminal, but for a completely differentclass of data. This is a great example of social filtering.

Anothergood example are prediction markets, where groups of people vote onwhat stock or movie or politician is likely to win — in some cases bybuying virtual stock in them — as a means to predict the future. Ithas been shown that prediction markets do a pretty good job of makingaccurate predictions in fact. In addition expertise referral serviceshelp people get answers to questions from communities of experts. Theseservices have been around in one form or another for decades and haverecently come back into vogue with services like Yahoo Answers. Amazonhas also taken a stab at this with their Amazon Mechanical Turk, whichenables “programs” to be constructed in which people perform the work.

I think social networking, social filtering, prediction markets,expertise referral networks, and collective collaboration are extremelyvaluable. By leveraging other people individuals and groups can stayahead of complexity and can also get the benefit of wide-areacollective cognition. These approaches to collective cognition arebeginning to filter into the processes of organizations and othercommunities. For example, there is recent interest in applying socialnetworking to niche communities and even enterprises.

The Semantic Webwill enrich all of these activities — making social networks andsocial filtering more productive. It’s not an either/or choice — thesetechnologies are extremely compatible in fact. By leveraging acommunity to tag, classify and organize content, for example, themeaning of that content can be collectively enriched. This is alreadyhappening in a primitive way in many social media services. TheSemantic Web will simply provide a richer framework for doing this.

The combination of the Semantic Web with emerging social networkingand social filtering will enable something greater than either on it’sown. Together, together these two technologies will enable much smarter groups, social networks, communities and organizations. But this still will not get us all the way past the Collective IQBarrier. It may get us close the threshold though. To cross thethreshold we will need to enable an even more powerful form ofcollective cognition.

The Agent Web

To cope with the enormous future scale andcomplexity of the Web, desktop and enterprise, each individual and group willreally need not just a single assistant, or even a community of humanassistants working on common information (a social filtering communityfor example), they will need thousands or millions of assistants working specificallyfor them. This really only becomes affordable and feasible if we canvirtualize what an “assistant” is.

Human assistants are at the top ofthe intelligence pyramid — they are extremely smart and powerful, and they are expensive — they should not beused for simple tasks like sorting content, that’s just a waste oftheir capabilities. It would be like using a supercomputer array tospellcheck a document. Instead, we need to free humans up to do thereally high-value information tasks, and find a way to farm out thelow-value, rote tasks to software. Software is cheap or even free and it can be replicated as much asneeded in order to parallelize. A virtual army of intelligent agents is less expensive than a single human assistant, and much more suited to sifting through millions of Web pages every day.

But where will these future intelligent agents get their intelligence? In past attempts at artificial intelligence, researchers tried to buildgigantic expert systems that could reason as well as a small child forexample. These attempts met with varying degrees of success, but theyall had one thing in common: They were monolithic applications.

I believe that that future intelligent agents should be simple. They should not be advanced AI programs or expert systems. They should be capable of a few simple behaviors, the most important of which is to reason against sets of rules and semantic data. The basic logic necessary for reasoning is not enormous and does not require any AI — it’s just the ability to follow logical rules and perhaps do set operations. They should be lightweight and highly mobile. Insteadof vast monolithic AI, I am talking about vast numbers of very simpleagents that working together can do emergent, intelligent operations en masse.

For example search — you might deploy a thousand agents to search all the sites about Italy for recipes and then assemble those results into a database instantaneously. Or you might dispatch a thousand or more agents to watch for a job that matches your skills and goals across hundreds of thousands or millions of Websites. They could watch and wait until jobs that matched your criteria appeared, and then they could negotiate amongst themselves to determine which of the possible jobs they found were good enough to show you. Another scenario might be commerce — you could dispatch agents to find you the best deal on a vacation package, and they could even negotiate an optimal itinerary and price for you. All you would have to do is choose between a few finalist vacation packages and make the payment. This could be a big timesaver.

The above examples illustrate how agents might help an individual, but how might they help a group or organization? Well for one thing agents could continuously organize and re-organize information for a group. They could also broker social interactions — for example, by connecting people to other people with matching needs or interests, or by helping people find experts who could answer their questions. One of the biggest obstacles to getting past the Collective IQ Barrier is simply that people cannot keep track of more than a few social relationships and information sources at aany given time — but with an army of agents helping them, individuals might be able to cope with more relationships and data sources at once; the agents would act as their filters, deciding what to let through and how much priority to give it. Agents can also help to make recommendations, and to learn to facilitate and even automate various processes such as finding a time to meet, or polling to make a decision, or escalating an issue up or down the chain of command until it is resolved.

To make intelligent agents useful, they will need access to domain expertise. But the agents themselves will not contain any knowledge or intelligence of their own. The knowledge will exist outside on the Semantic Web, and so will the intelligence. Their intelligence, like their knowledge, will be externalized and virtualized in the form of axioms or rules that will exist out on the Web just like web pages.

For example, a set of axioms about travel could be published to the Web in the form of a document that formally defined them. Any agent that needed to process travel-related content could reference these axioms in order to reason intelligently about travel in the same way that it might reference an ontology about travel in order to interpret travel data structures. The application would not have to be specifically coded to know about travel — it could be a generic simple agent — but whenever it encountered travel-related content it could call up the axioms about travel from the location on the Web where they were hosted, and suddenly it could reason like an expert travel agent. What’s great about this is that simple generic agents would be able to call up domain expertise on an as-needed basis for just about any domain they might encounter. Intelligence — the heuristics, algorithms and axioms that comprise expertise, would be as accessible as knowledge — the data and connections between ideas and information on the Web.

The axioms themselves would be created by human experts in various domains, and in some cases they might even be created or modified by agents as they learned from experience. These axioms might be provided for free as a public service, or as fee-based web-services via API’s that only paying agents could access.

The key is that model is extremely scaleable — millions or billions of axioms could be created, maintained, hosted, accessed, and evolved in a totally decentralized and parallel manner by thousands or even hundreds of thousands of experts all around the Web. Instead of a few monolithic expert systems, the Web as a whole would become a giant distributed system of experts. There might be varying degrees of quality among competing axiom-sets available for any particular domain, and perhaps a ratings system could help to filter them over time. Perhaps a sort of natural selection of axioms might take place as humans and applications rated the end-results of reasoning using particular sets of axioms, and then fed these ratings back to the sources of this expertise, causing them to get more or less attention from other agents in the future. This process would be quite similar to the human-level forces of intellectual natural-selection at work in fields of study where peer-review and competition help to filter and rank ideas and their proponents.

Virtualizing Intelligence

What I have been describing is the virtualization of intelligence — making intelligence and expertise something that can be “published” to the Web and shared just like knowledge, just like an ontology, a document, a database, or a Web page. This is one of the long-term goals of the Semantic Web and it’s already starting now via new languages, such as SWRL, that are being proposed for defining and publishing axioms or rules to the Web. For example, “a non-biologicalparent of a person is their step-parent” is asimple axiom. Another axiom might be, “A child of a sibling of your parent is your cousin.” Using such axioms, an agent could make inferences and do simple reasoning about social relationships for example.

SWRL and other proposed rules languages provide potentialopen-standards for defining rules and publishing them to the Web sothat other applications can use them. By combining these rules withrich semantic data, applications can start to do intelligent things,without actually containing any of the intelligence themselves. The intelligence– the rules and data — can live “out there” on the Web, outside the code of various applications.

All theapplications have to know how to do is find relevant rules, interpret them, and apply them. Even the reasoning that may be necessary can be virtualized into remotely accessible Web services so applications don’t even have to do that part themselves (although many may simply include open-source reasoners in the same way that they include open-source databases or search engines today).

In other words, just as HTML enables any app to process and formatany document on the Web, SWRL + RDF/OWL may someday enable any application to reasonabout what the document discusses. Reasoning is the last frontier. Byvirtualizing reasoning — the axioms that experts use to reason aboutdomains — we can really begin to store the building blocks of humanintelligence and expertise on the Web in a universally-accessibleformat. This to me is when the actual “Intelligent Web” (what I callWeb 4.0) will emerge.

The value of this for groups and organizations is that they can start to distill their intelligence from individuals that comprise them into a more permanent and openly accessible form — axioms that live on the Web and can be accessed by everyone. For example, a technical support team for a product learns many facts and procedures related to their product over time. Currently this learning is stored as knowledge in some kind of tech support knowledgebase. But the expertise for how to find and apply this knowledge still resides mainly in the brains of the people who comprise the team itself.

The Semantic Web provides ways to enrich the knowledgebase as well as to start representing and saving the expertise that the people themselves hold in their heads, in the form of sets of axioms and procedures. By storing not just the knowledge but also the expertise about the product, the humans on the team don’t have to work as hard to solve problems — agents can actually start to reason about problems and suggest solutions based on past learning embodied in the common set of axioms. Of course this is easier said than done — but the technology at least exists in nascent form today. In a decade or more it will start to be practical to apply it.

Group Minds

Someday in the not-too-distant-future groups will be able toleverage hundreds or thousands of simple intelligent agents. Theseagents will work for them 24/7 to scour the Web, the desktop, theenterprise, and other services and social networks they are related to. They will help both the individuals as well as the collectives as-a-whole. They willbe our virtual digital assistants, always alert and looking for thingsthat matter to us, finding patterns, learning on our behalf, reasoningintelligently, organizing our information, and then filtering it,visualizing it, summarizing it, and making recommendations to us sothat we can see the Big Picture, drill in wherever we wish, and makedecisions more productively.

Essentially these agents will give groups something like their own brains. Today the only brains in a group reside in the skulls of the people themselves. But in the future perhaps we will see these technologies enable groups to evolve their own meta-level intelligences: systems of agents reasoning on group expertise and knowledge.

This will be a fundamental leap to a new order of collective intelligence. For the first time groups will literally have minds of their own, minds that transcend the mere sum of the individual human minds that comprise their human, living facets. I call these systems “Group Minds” and I think they are definitely coming. In fact there has been quite a bit of research on the subject of facilitating group collaboration with agents, for example, in government agencies such as DARPA and the military, where finding ways to help groups think more intelligently is often a matter of life and death.

The big win from a future in which individuals and groups canleverage large communities of intelligent agents is that they will bebetter able to keep up with the explosive growth of information complexity andsocial complexity. As the saying goes, “it takes a village.” There is just too much information, and too many relationships, changing too fast and this is only going to get more intense in years to come. The only way to cope with such a distributed problem is a distributed solution.

Perhaps by 2030 it will not be uncommon for Individuals and groups to maintain largenumbers of virtual assistants — agents that will help them keep abreast of themassively distributed, always growing and shifting information and sociallandscapes. When you really think about this, how else could we eversolve this? This is really the only practical long-term solution. But today it is still a bit of a pipedream; we’re not there yet. The key however is that we are closer than we’ve ever been before.

Conclusions

The Semantic Web provides the key enabling technology for all ofthis to happen someday in the future. By enriching the content of theWeb it first paves the way to a generation of smarter applications andmore productive individuals, groups and organizations.

The next majorleap will be when we begin to virtualize reasoning in the form ofaxioms that become part of the Semantic Web. This will enable a newgeneration of applications that can reason across information andservices. This will ultimately lead to intelligent agents that will be able to assist individuals,groups, social networks, communities, organizations and marketplaces sothat they can remain productive in the fact of the astonishinginformation and social network complexity in our future.

By adding more knowledge into our information, the Semantic Webmakes it possible for applications (and people) to use information moreproductively. By adding more intelligence between people, information,and applications, the Semantic Web will also enable people andapplications to become smarter. In the future, these more-intelligentapps will facilitate higher levels of individual and collectivecognition by functioning as virtual intelligent assistants forindividuals and groups (as well as for online services).

Once we begin to virtualize not just knowledge (semantics) but alsointelligence (axioms) we will start to build Group Minds — groups that have primitive minds of their own. When we reach this point we will finally enable organizations to breakpast the Collective IQ Barrier: Organizations will start to becomesmarter than the sum of their parts. The intelligence of anorganization will not just be from its people, it will also come fromits applications. The number of intelligent applications in anorganization may outnumber the people by 1000 to 1, effectivelyamplifying each individual’s intelligence as well as the collectiveintelligence of the group.

Because software agents work all the time,can self-replicate when necessary, and are extremely fast and precise,they are ideally-suited to sifting in parallel through the millions or billions ofdata records on the Web, day in and day out. Humans and even groups ofhumans will never be able to do this as well. And that’s not what theyshould be doing! They are far too intelligent for that kind of work.Humans should be at the top of the pyramid, making the decisions,innovating, learning, and navigating.

When we finally reach this stage where networks of humans and smartapplications are able to work together intelligently for common goals,I believe we will witness a real change in the way organizations arestructured. In Group Minds, hierarchy will not be as necessary — the maximum effectivesize of a human Group Mind will be perhaps in the thousands or even themillions instead of around 50 people. As a result the shape of organizations in thefuture will be extremely fluid, and most organizations will be flat orcontinually shifting networks. For more on this kind of organization,read about virtual teams and networking, such as these books (by friends of mine who taught me everything I know about network-organization paradigms.)

I would also like to note that I am not proposing “strong AI” — a vision in which we someday makeartificial intelligences that are as or more intelligent thanindividual humans. I don’t think intelligent agents will individually be very intelligent. It will only be in vast communities of agents that intelligence will start to emerge. Agents are analogous to the neurons in the human brain — they really aren’t very powerful on their own.

I’m also not proposing that Group Minds will beas or more intelligent as the individual humans in groups anytime soon. I don’t think thatis likely in our lifetimes. The cognitive capabilities of an adult human are the product of millions of years of evolution. Even in the accelerated medium of the Web where evolution can take place much faster in silico, it may still take decades or even centuries to evolve AI that rivals the human mind (and I doubt such AI will ever be truly conscious, which means that humans, with their inborn natural consciousness, may always play a special and exclusive role in the world to come, but that is the subject of a different essay). But even if they will not be as intelligent as individual humans, Ido think that Group Minds, facilitated by masses of slightly intelligent agents and humans working in concert, can goa long way in helping individuals and groups become more productive.

It’s important to note that the future I am describing is notscience-fiction, but it also will not happen overnight. It will take atleast several decades, if not longer. But with the seeminglyexponential rate of change of innovation, we may make very large stepsin this direction very soon. It is going to be an exciting lifetime forall of us.

Scientists Encode Message into Bacterial DNA

Nova — Fri, 02 Mar 2007 03:36:18 +0000

Japanese scientists have developed a technique that can encode 100-bit messages into the DNA of common bacteria. The bacteria replicate and pass the message down from generation to generation for at least thousands of years. Because there are millions or more copies of the message it can survive gradual degradation or mutuations (so they claim). Perhaps by taking a sample of the message across a large number of descendant bacteriums any errors or mutations can be detected and corrected. The message that was encoded was ""e=mc² 1905".

I’ve written about the potential of storing messages in DNA in the past here, and here.

What’s interesting of course is that since this is possible it begs the question of whether there are already messages encoded into the DNA of various living things on Earth? We might want to look at E Coli, or other common organisms, or perhaps human, dolphin, and whale DNA. We might also want to look at birds and lizards since they come down more directly from dinosaurs. Who knows — maybe a long long time ago someone left us messages there, or their signature at least.

There are two places that I think it is most likely that we will first receive messages from aliens, if we ever do:

Our own DNA (or that of other living species on Earth)
The Internet. It’s the logical place to establish communication with us. Perhaps via a Myspace page…

Intelligence is in the Connections

Nova — Tue, 20 Feb 2007 16:26:58 +0000

Google’s Larry Page recently gave a talk to the AAAS about how Google is looking towards a future in which they hope to implement AI on a massive scale. Larry’s idea is that intelligence is a function of massive computation, not of “fancy whiteboard algorithms.” In other words, in his conception the brain doesn’t do anything very sophisticated, it just does a lot of massively parallel number crunching. Each processor and its program is relatively “dumb” but from the combined power of all of them working together “intelligent” behaviors emerge.

Larry’s view is, in my opinion, an oversimplification that will not lead to actual AI. It’s certainly correct that some activities that we call “intelligent” can be reduced to massively parallel simple array operations. Neural networks have shown that this is possible — they excel at low level tasks like pattern learning and pattern recognition for example. But neural networks have not proved capable of higher level cognitive tasks like mathematical logic, planning, or reasoning. Neural nets are theoretically computationally equivalent to Turing Machines, but nobody (to my knowledge) has ever succeeded in building a neural net that can in practice even do what a typical PC can do today — which is still a long way short of true AI!

Somehow our brains are capable of basic computation, pattern detection and learning, simple reasoning, and advanced cognitive processes like innovation and creativity, and more. I don’t think that this richness is reducible to massively parallel supercomputing, or even a vast neural net architecture. The software — the higher level cognitive algorithms and heuristics that the brain “runs” — also matter. Some of these may be hard-coded into the brain itself, while others may evolve by trial-and-error, or be programmed or taught to it socially through the process of education (which takes many years at the least).

Larry’s view is attractive but decades of neuroscience and cognitive science have shown conclusively that the brain is not nearly as simple as we would like it to be. In fact the human brain is far more sophisticated than any computer we know of today, even though we can think of it in simple terms. It’s a highly sophisticated system comprised of simple parts — and actually, the jury is still out on exactly how simple the parts really are — much of the computation in the brain may be sub-neuronal, meaning that the brain may actually a much much more complex system than we think.

Perhaps the Web as a whole is the closest analogue we have today for the brain — with millions of nodes and connections. But today the Web is still quite a bit smaller and simpler than a human brain. The brain is also highly decentralized and it is doubtful than any centralized service could truly match its capabilities. We’re not talking about a few hundred thousand linux boxes — we’re talking about hundreds of billions of parallel distributed computing elements to model all the neurons in a brain, and this number gets into the trillions if we want to model all the connections. The Web is not this big, and neither is Google.

One reader who commented on Larry’s talk made an excellent point on what this missing piece may be: “Intelligence is in the connections, not the bits.”The point is that most of the computation in the brain actually takesplace via the connections between neurons, regions, and perhapsprocesses. This writer also made some good points about quantumcomputation and how the brain may make use of it, a view that forexample Roger Penrose and others have spent a good deal of time on.There is some evidence that brain may make use of microtubules andquantum-level computing. Quantum computing is inherently about fields,correlations and nonlocality. In other words the connections in thebrain may exist on a quantum level, not just a neurological level.

Whether quantum computation is the key or not still remains to bedetermined. But regardless, essentially, Larry’s approach is equivalentto just aiming a massively parallel supercomputer at the Web and hopingthat will do the trick. Larry mentions for example that if allknowledge exists on the Web you should be able to enter a query and geta perfect answer. In his view, intelligence is basically just search ona grand scale. All answers exist on the Web, and the task is just tomatch questions to the right answers. But wait? Is that all thatintelligence does? Is Larry’s view too much of an oversimplification?Intelligence is not just about learning and recall, it’s also aboutreasoning and creativity. Reasoning is not just search. It’s unclearhow Larry’s approach would address that.

In my own opinion, for global-scale AI to really emerge the Web has toBE the computer. The computation has to happen IN the Web, betweensites and along connections — rather than from outside the system. Ithink that is how intelligence will ultimately emerge on a Web-widescale. Instead of some Google Godhead implementing AI from afar for thewhole Web, I think it is more likely that every site, app and person onthe Web will help to implement it. It will be much more of a hybridsystem that combines decentralized human and machine intelligences andtheir interactions along data connections and social relationships. Ithink this may emerge from a future evolution of the Web that providesfor much richer semantics on every piece of data and hyperlink on theWeb, and for decentralized learning, search, and reasoning to takeplace within every node on the Web. I think the Semantic Web is anecessary technology for this to happen, but it’s only the first step.More will need to happen on top of it for this vision to reallymaterialize.

My view is more of an “agent metaphor” for intelligence — perhaps itis similar to Marvin Minsky’s Society of Mind ideas. I think that mindsare more like communities than we presently think. Even in our ownindividual minds for example we experience competing thoughts, multiplethreads, and a kind of internal ecology and natural selection of ideas.These are not low-level processes — they are more like agents — theyare actually each somewhat “intelligent” on their own, they seem to besomewhat autonomous, and they interact in intelligent almost socialways.

Ideas seem to be actors, not just passive data points — they arecompeting for resources and survival in a complex ecology that existsboth within our individual minds and between them in socialrelationships and communities. As the theory of memetics proposes,ideas can even transport themselves through language, culture, andsocial interactions in order to reproduce and evolve from mind to mind.It is an illusion to think that there is some central self or “I” thatcontrols the process (that is just another agent in the community infact, perhaps one with a kind of reporting and selection role).

I’m not sure the complex social dynamics of these communities ofintelligence can really be modeled by a search engine metaphor. Thereis a lot more going on than just search. As well as communication andreasoning between different processes, there may in fact be feedbackacross levels from the top-down as well as the from the bottom-up.Larry is essentially proposing that intelligence is a purely bottom-upemergent process that can be reduced to search in the ideal, simplestcase. I disagree. I think there is so much feedback in every directionthat medium and the content really cannot be separated. The thoughtsthat take place in the brain ultimately feedback down to the neuralwetware itself, changing the states of neurons and connections –computation flows back down from the top, it doesn’t only flow up fromthe bottom. Any computing system that doesn’t include this kind offeedback in its basic architecture will not be able to implement trueAI.

In short, Google is not the right architecture to truly build a globalbrain on. But it could be a useful tool for search andquestions-and-answers in the future, if they can somehow keep up withthe growth and complexity of the Web.

New Simulation Explains Why Extraterrestrial Life Hasn't Found Us Yet

Nova — Fri, 19 Jan 2007 16:07:03 +0000

Interesting new study…

It ranks among the most enduring mysteries of the cosmos. Physicists
call it the Fermi paradox after the Italian Nobel laureate Enrico
Fermi, who, in 1950, pointed out the glaring conflict between
predictions that life was elsewhere in the universe – and the
conspicuous lack of aliens who have come to visit.

Now a Danish
researcher believes he may have solved the paradox. Extra-terrestrials
have yet to find us because they haven’t had enough time to look.

(via Kurzweil)

My guess is that the numbers might be different however. An alien civilization that could send out probes at one tenth the speed of light would probably be smart enough to create self-replicating probes, in order to generate thousands or millions of probes over time. This might bring the numbers down significantly — although perhaps still not enough.

Must-Know Terms for the 21st Century Intellectual

Nova — Fri, 12 Jan 2007 15:24:13 +0000

Read this fun article that lists and defines some of the key concepts that every post-singularity transhumanist meta-intellectual should know! (via Kurzweil)

British Ministry of Defense Chief Resigns; Cites Concerns About UFO's

Nova — Fri, 10 Nov 2006 06:30:00 +0000

Ok, here’s a very unusual news item:

During his time as head of the Ministry of Defence UFO project, Nick
Pope was persuaded into believing that other lifeforms may visit Earth
and, more specifically, Britain.

His concern is that "highly credible" sightings are simply dismissed.

And he complains that the project he once ran is now "virtually closed" down, leaving the country "wide open" to aliens.

Mr Pope decided to speak out about his worries after resigning
from his post at the Directorate of Defence Security at the MoD this
week.

"The consequences of getting this one wrong could be huge," he said.

Read the rest here. I have several thoughts about this news and what it might mean…

Mr. Pope has been speaking out on these issues for years. What’s
most interesting is that the MoD allowed him to. This seems to tacitly
confirm that what he’s saying may have some truth to it. If he was
actually wrong or a lunatic they would have fired him long ago, or had
him conveniently committed to an asylum. Since they didn’t fire him,
this would seem to indicate that the higher-ups must have at least
enough respect for him to keep him on as one of their chiefs (or maybe
he had some kind of government job that he couldn’t be fired from?).

In any case, I wonder why he thinks aliens would have bad
intentions? Perhaps he has information that is not available to the
public. But if aliens exist, and if they have such advanced
technologies, and if they have bad intentions, wouldn’t they have
invaded long ago? What would be their reason to wait if their intention
was to invade? Waiting would only give us more time to develop
technologies to fight them with.

The fact that, despite quite a lot of evidence of UFO’s, they have
NOT invaded, seems to indicate that their intentions are not harmful.
Perhaps they are here to study us, or protect us from ourselves, or to
refuel and re-provision enroute to somewhere else? Or maybe they are
missionaries? Or maybe they are not interested in humans at all. Maybe
it’s the cats they are here to talk to. Let’s face it — the cats are
clearly much cooler than the humans.

Then again, this reminds me of an old Twilight Zone episode from the
early 1960’s that I read about in which aliens landed and stated that
their sole purpose for coming to earth was for the betterment of
humanity. They had a special text of principles entitled, "To Serve
Man." It wasn’t until the end of the episode that it was revealed that
this text was actually a cookbook.

Minding The Planet — The Meaning and Future of the Semantic Web

Nova — Mon, 06 Nov 2006 11:34:13 +0000

NOTES

Master Copy can be found at this URL or http://tinyurl.com/yynb93
Last Update: Tuesday, November 7, 2006, 10:17AM PST
License — This article is distributed under the Creative Commons Deed. If you would like to distribute a version of thisarticle, please link back to http://www.mindingtheplanet.net from yourversion, thanks.
Printable version — Click here to download the printable PDF version of this article
Illustrated Version — See the version by the Lifeboat Foundation

Prelude

Many years ago, in the late 1980s, while I was still a college student, I visited my late grandfather, Peter F. Drucker, at his home in Claremont, California. He lived near the campus of Claremont College where he was a professor emeritus. On that particular day, I handed him a manuscript of a book I was trying to write, entitled, “Minding the Planet” about how the Internet would enable the evolution of higher forms of collective intelligence.

My grandfather read my manuscript and later that afternoon we sat together on the outside back porch and he said to me, “One thing is certain: Someday, you will write this book.” We both knew that the manuscript I had handed him was not that book, a fact that was later verified when I tried to get it published. I gave up for a while and focused on college, where I was studying philosophy with a focus on artificial intelligence. And soon I started working in the fields of artificial intelligence and supercomputing at companies like Kurzweil, Thinking Machines, and Individual.

A few years later, I co-founded one of the early Web companies, EarthWeb, where among other things we built many of the first large commercial Websites and later helped to pioneer Java by creating several large knowledge-sharing communities for software developers. Along the way I continued to think about collective intelligence. EarthWeb and the first wave of the Web came and went. But this interest and vision continued to grow. In 2000 I started researching the necessary technologies to begin building a more intelligent Web. And eventually that led me to start my present company, Radar Networks, where we are now focused on enabling the next-generation of collective intelligence on the Web, using the new technologies of the Semantic Web.

But ever since that day on the porch with my grandfather, I remembered what he said: “Someday, you will write this book.” I’ve tried many times since then to write it. But it never came out the way I had hoped. So I tried again. Eventually I let go of the book form and created this weblog instead. And as many of my readers know, I’ve continued to write here about my observations and evolving understanding of this idea over the years. This article is my latest installment, and I think it’s the first one that meets my own standards for what I really wanted to communicate. And so I dedicate this article to my grandfather, who inspired me to keep writing this, and who gave me his prediction that I would one day complete it.

This is an article about a new generation of technology that is sometimes called the Semantic Web, and which could also be called the Intelligent Web, or the global mind. But what is the Semantic Web, and why does it matter, and how does it enable collective intelligence? And where is this all headed? And what is the long-term far future going to be like? Is the global mind just science-fiction? Will a world that has a global mind be good place to live in, or will it be some kind of technological nightmare?

I’ve often joked that it is ironic that a term that contains theword “semantic” has such an ambiguous meaning for most people. Mostpeople just have no idea what this means, they have no context for it,it is not connected to their experience and knowledge. This is aproblem that people who are deeply immersed in the trenches of theSemantic Web have not been able to solve adequately — they have notfound the words to communicate what they can clearly see, what they areworking on, and why it matters for everyone. In this article I havetried, and hopefully succeeded, in providing a detailed introductionand context for the Semantic Web fornon-technical people. But even technical people working in the fieldmay find something of interest here as I piece together the fragmentsinto a Big Picture and a vision for what might be called “Semantic Web2.0.”

I hope the reader will bear with me as Ibounce around across different scales of technology and time, and fromthe extremes of core technology to wild speculation in order to tellthis story. If you are looking for the cold hardscience of it all, this article will provide an understanding but willnot satisfy your need for seeing the actual code; there are otherplaceswhere you can find that level of detail and rigor. But if you want tounderstand what it all really means and what the opportunity and futurelookslike – this may be what you are looking for.

I should also note that all of this is my personal view of what I’vebeen working on,and what it really means to me. It is not necessarily the official viewof the mainstream academic Semantic Web community — although there arecertainly many places where we all agree. But I’m sure that somereaders will certainly disagree or raise objections to some of myassertions, and certainly to my many far-flung speculations about thefuture. I welcome those different perspectives; we’re all trying tomake sense of this and the more of us who do that together, the more wecan collectively start to really understand it. So please feel free towrite your own vision or response, and please let me know so I can linkto it!

So with this Prelude in mind, let’s get started…

The Semantic Web Vision

The Semantic Web is a set of technologies which are designed toenable aparticular vision for the future of the Web – a future in which allknowledge exists on the Web in a format that software applications canunderstand andreason about. By making knowledge more accessible to software, softwarewillessentially become able to understand knowledge, think about knowledge,and createnew knowledge. In other words, software will be able to be moreintelligent –not as intelligent as humans perhaps, but more intelligent than say,your wordprocessor is today.

The dream of making software more intelligent has been around almost as longas software itself. And although it is taking longer to materialize than past experts hadpredicted, progress towards this goal is being steadilymade. At the same time, the shape of this dream is changing. It is becomingmore realistic and pragmatic. The original dream of artificial intelligence wasthat we would all have personal robot assistants doing all the work we don’twant to do for us. That is not the dream of the Semantic Web. Instead, today’sSemantic Web is about facilitating what humans do – it is about helping humansdo things more intelligently. It’s not a vision in which humans do nothing andsoftware does everything.

The Semantic Web vision is not just about helping software become smarter –it is about providing new technologies that enable people, groups,organizations and communities to be smarter.

For example, by providing individuals with tools that learn about what theyknow, and what they want, search can be much more accurate and productive.

Using software that is able to understand and automatically organize largecollections of knowledge, groups, organizations and communities can reachhigher levels of collective intelligence and they can cope with volumes ofinformation that are just too great for individuals or even groups tocomprehend on their own.

Another example: more efficient marketplaces can be enabled by software thatlearns about products, services, vendors, transactions and market trends andunderstands how to connect them together in optimal ways.

In short, the Semantic Web aims to make software smarter, not just for itsown sake, but in order to help make people, and groups of people, smarter. Inthe original Semantic Web vision this fact was under-emphasized, leading to theimpression that Semantic Web was only about automating the world. In fact, it isreally about facilitating the world.

The Semantic Web Opportunity

The Semantic Web is one of the most significant things to happen since theWeb itself. But it will not appear overnight. It will take decades. It willgrow in a bottom-up, grassroots, emergent, community-driven manner just likethe Web itself. Many things have to converge for this trend to really take off.

The core open standards already exist, but the necessary development tools haveto mature, the ontologies that define human knowledge have to come into beingand mature, and most importantly we need a few real “killer apps” to prove thevalue and drive adoption of the Semantic Web paradigm. The first generation ofthe Web had its Mozilla, Netscape, Internet Explorer, and Apache – and it alsohad HTML, HTTP, a bunch of good development tools, and a few killer apps andservices such as Yahoo! and thousands of popular Web sites. The same things arenecessary for the Semantic Web to take off.

And this is where we are today – this all just about to start emerging.There are several companies racing to get this technology, or applications ofit, to market in various forms. Within a year or two you will see mass-consumerSemantic Web products and services hit the market, and within 5 years therewill be at least a few “killer apps” of the Semantic Web. Ten years from nowthe Semantic Web will have spread into many of the most popular sites andapplications on the Web. Within 20 years all content and applications on theInternet will be integrated with the Semantic Web. This is a sea-change. A bigevolutionary step for the Web.

The Semantic Web is an opportunity to redefine, or perhaps to better define,all the content and applications on the Web. That’s a big opportunity. Andwithin it there are many business opportunities and a lot of money to be made. It’snot unlike the opportunity of the first generation of the Web. There areplatform opportunities, content opportunities, commerce opportunities, searchopportunities, community and social networking opportunities, and collaborationopportunities in this space. There is room for a lot of players to compete andat this point the field is wide open.

The Semantic Web is a blue ocean waiting to be explored. And like anyunexplored ocean its also has its share of reefs, pirate islands, hidden treasure, shoals,whirlpools, sea monsters and typhoons. But there are new worlds out there to be discovered,and they exert an irresistible pull on the imagination. This is an excitingfrontier – and also one fraught with hard technical and social challenges thathave yet to be solved. For early ventures in the Semantic Web arena, it’s notgoing to be easy, but the intellectual and technological challenges, and the potentialfinancial rewards, glory, and benefit to society, are worth the effort andrisk. And this is what all great technological revolutions are made of.

Semantic Web 2.0

Some people who have heard the term “Semantic Web” thrown around too muchmay think it is a buzzword, and they are right. But it is not just a buzzword –it actually has some substance behind it. That substance hasn’t emerged yet,but it will. Early critiques of the Semantic Web were right – the early visiondid not leverage concepts such as folksonomy and user-contributed content atall. But that is largely because when the Semantic Web was originally conceivedof Web 2.0 hadn’t happened yet. The early experiments that came out of researchlabs were geeky, to put it lightly, and impractical, but they are already beingfollowed up by more pragmatic, user-friendly approaches.

Today’s Semantic Web – what we might call “Semantic Web 2.0” is a kinder,gentler, more social Semantic Web. It combines the best of the original visionwith what we have all learned about social software and community in the last10 years. Although much of this is still in the lab, it is already starting totrickle out. For example, recently Yahoo! started a pilot of the Semantic Webbehind their food vertical. Other organizations are experimenting with usingSemantic Web technology in parts of their applications, or to store or mapdata. But that’s just the beginning.

The Google Factor

Entrepreneurs, venture capitalists and technologists are increasinglystarting to see these opportunities. Who will be the “Google of the SemanticWeb?” – will it be Google itself? That’s doubtful. Like any entrenchedincumbent, Google is heavily tied to a particular technology and worldview. Andin Google’s case it is anything but semantic today. It would be easier for anupstart to take this position than for Google to port their entireinfrastructure and worldview to a Semantic Web way of thinking.

If it is goingto be Google it will most likely be by acquisition rather than by internal origination. Andthis makes more sense anyway – for Google is in a position where they can just wait and buy the winner,at almost any price, rather than competing in the playing field. One thing to note however is that Google has at least one product offering that shows some potential for becoming a key part of the Semantic Web. I am speaking of Google Base, Google’s open database which is meant to be a registry for structured data so that it can be found in Google search. But Google Base does not conform to or make use of the many open standards of the Semantic Web community. That may or may not be a good thing, depending on your perspective.

Of course the downside of Google waiting to join the mainstream Semantic Web community until after the winner is announced is very large – once there is a winner it may be too late for Google to beat them. Thewinner of the Semantic Web race could very well unseat Google. The strategistsat Google are probably not yet aware of this but as soon as they seesignificant traction around a major Semantic Web play it will become of interestto them.

In any case, I think there won’t be just one winner, there will be severalmajor Semantic Web companies in the future, focusing on different parts of theopportunity. And you can be sure that if Google gets into the game, every majorportal will need to get into this space at some point or risk becomingirrelevant. There will be demand and many acquisitions. In many ways the Semantic Web will not be controlled by just one company — it will be more like a fabric that connects them all together.

Context is King — The Nature ofKnowledge

It should be clear by now that the Semantic Web is all about enablingsoftware (and people) to work with knowledge more intelligently. But what isknowledge? Knowledge is not just information. It is meaningful information – itis information plus context. For example, if I simply say the word “sem” toyou, it is just raw information, it is not knowledge. It probably has nomeaning to you other than a particular set of letters that you recognize and asound you can pronounce, and the mere fact that this information was stated byme.

But if I tell you that “sem” it is the Tibetan word for “mind” then suddenly,“sem means mind in Tibetan” to you. If I further tell you that Tibetans have about as many words for “mind” as Eskimos have for “snow,” this is further meaning. Thisis context, in other words, knowledge, about the sound “sem.” The sound is raw information. When it is given context itbecomes a word, a word that has meaning, a word that is connected to conceptsin your mind – it becomes knowledge. By connecting raw information to context,knowledge is formed.

Once you have acquired a piece of knowledge such as “sem means mind in Tibetan,” you may then also form further knowledgeabout it. For example, you may form the memory, “Nova said that ‘sem means mind in Tibetan.’” You mightalso connect the word “sem” to networks of further concepts you have about Tibet and your understanding of what the word “mind” means.

The mind is the organ of meaning – mind is where meaning is stored,interpreted and created. Meaning is not “out there” in the world, it is purelysubjective, it is purely mental. Meaning is almost equivalent to mind in fact.For the two never occur separately. Each of our individual minds has some way of internally representing meaning — when we read or hear a word that we know, our minds connect that to a network of concepts about it and at that moment it means something to us.

Digging deeper, if you are really curious,or you happen to know Greek, you may also find that a similar sound occurs inthe Greek word, sēmantikós – which means “having meaning” and in turn is the root of the English word “semantic”which means “pertaining to or arising from meaning.” That’s an odd coincidence!“Sem” occurs in Tibetan word for mind, and the English and Greek words that allrelate to the concepts of “meaning” and “mind.” Even stranger is that not only do these words have a similar sound, they have a similar meaning.

With all this knowledge at yourdisposal, when you then see the term “Semantic Web” you may be able to inferthat it has something to do with adding “meaning” to the Web. However, if youwere a Tibetan, perhaps you might instead think the term had something to dowith adding “mind” to the Web. In either case you would be right!

Discovering New Connections

We’ve discovered a new connection — namely that there is an implicit connectionbetween “sem” in Greek, English and Tibetan: they all relate to meaning andmind. It’s not a direct, explicit connection – it’s not evident unless you digfor it. But it’s a useful tidbit of knowledge once it’s found. Unlike the direct migration of the sound “sem” from Greek to English,there may not have ever been a direct transfer of this sound from Greek toSanskrit to Tibetan. But in a strange and unexpected way, they are all connected. This connectionwasn’t necessarily explicitly stated by anyone before, but was uncovered byexploring our network of concepts and making inferences.

The sequence of thought about “sem”above is quite similar to kind of intellectual reasoning and discovery that theactual Semantic Web seeks to enable software to do automatically. How is this kind of reasoning and discovery enabled? The Semantic Web providesa set of technologies for formally defining the context of information. Just asthe Web relies on a standard formal specification for “marking up” informationwith formatting codes that enable any applications that understand those codesto format the information in the same way, the Semantic Web relies on newstandards for “marking up” information with statements about its context – itsmeaning – that enable any applications to understand, and reason about, the meaning of those statements in the same way.

By applying semantic reasoning agents to large collections of semantically enhanced content, all sorts of new connections may be inferred, leading to new knowledge, unexpected discoveries and useful additional context around content. This kind of reasoning and discovery is already taking place in fields from drug discovery and medical research, to homeland security and intelligence. The Semantic Web is not the only way to do this — but it certainly will improve the process dramatically. And of course, with this improvement will come new questions about how to assess and explain how various inferences were made, and how to protect privacy as our inferencing capabilities begin to extend across ever more sources of public and private data. I don’t have the answers to these questions, but others are working on them and I have confidence that solutions will be arrived at over time.

Smart Data

By marking up information with metadata that formally codifies its context, we can make the data itself “smarter.” The data becomes self-describing. When you get a piece of data you also get the necessary metadata for understanding it. For example, if I sent you a document containing the word “sem” in it, I could add markup around that word indicating that it is the word for “mind” in the Tibetan language.

Similarly, a document containing mentions of “Radar Networks” could contain metadata indicating that “Radar Networks” is an Internet company, not a product or a type of radar technology. A document about a person could contain semantic markup indicating that they are residents of a certain city, experts on Italian cooking, and members of a certain profession. All of this could be encoded as metadata in a form that software could easily understand. The data carries more information about its own meaning.

The alternative to smart data would be for software to actually read and understand natural language as well as humans. But that’s really hard. To correctly interpret raw natural language, software would have to be developed that knew as much as a human being. But think about how much teaching and learning is required to raise a human being to the point where they can read at an adult level. It is likely that similar training would be necessary to build software that could do that. So far that goal has not been achieved, although some attempts have been made. While decent progress in natural language understanding has been made, most software that can do this is limited around particular vertical domains, and it’s brittle — it doesn’t do a good job of making sense of terms and forms of speech that it wasn’t trained to parse and make sense of.

Instead of trying to make software a million times smarter than it is today, it is much easier to just encode more metadata about what our information means. That turns out to be less work in the end. And there’s an added benefit to this approach — the meaning exists with the data and travels with it. It is independent of any one software program — all software can access it. And because the meaning of information is stored with the information itself, rather than in the software, the software doesn’t have to be enormous to be smart. It just has to know the basic language for interpreting the semantic metadata it finds on the information it works with.

Smart data enables relatively dumb software to be smarter with less work. That’s an immediate benefit. And in the long-term as software actually gets smarter, smart data will make it easier for it to start learning and exploring on its own. So it’s a win-win approach. Start with by adding semantic metadata to data, end up with smarter software.

Making Statements About the World

Metadata comes down to making statements about the world in a manner that machines, and perhaps even humans, can understand unambiguously. The same piece of metadata should be interpreted in the same way by different applications and readers.

There are many kinds of statementsthat can be made about information to provide it with context. For example, youcan state a definition such as “person” means “a human being or a legalentity.” You can state an assertion such as “Sue is a human being.” You canstate a rule such that “if x is a human being, then x is a person.”

From thesestatements it can then be inferred that “Sue is a person.” This inference is soobvious to you and me that it seems trivial, but most software today cannot dothis. It doesn’t know what a person is, let alone what a name is. But ifsoftware could do this, then it could for example, automatically organizedocuments by the people they are related to, or discover connections betweenpeople who were mentioned in a set of documents, or it could find documentsabout people who were related to particular topics, or it could give you a listof all the people mentioned in a set of documents, or all the documents relatedto a person.

Of course this is a very basicexample. But imagine if your software didn’t just know about people – it knewabout most of the common concepts that occur in your life. Your software wouldthen be able to help you work with your documents just about as intelligentlyas you are able to do by yourself, or perhaps even more intelligently, becauseyou are just one person and you have limited time and energy but your softwarecould work all the time, and in parallel, to help you.

Examples and Benefits

How could the existence of the Semantic Web and all the semantic metadata that defines it be really useful toeveryone in the near-term?

Well, for example, the problem of email spam would finally be cured:your software would be able to look at a message and know whether it wasmeaningful and/or relevant to you or not.

Similarly, you would never have to file anything by hand again. Your software could atuomate all filing and information organization tasks for you because it would understand your information and your interests. It would be able to figure out when to file something in a single folder, multiple folders, or new ones. It would organize everything — documents, photos, contacts, bookmarks, notes, products, music, video, data records — and it would do it even better and more consistently than you could on your own. Your software wouldn’t just organize stuff, it would turn it into knowledge by connecting it to more context. It could this not just for individuals, but for groups, organizations and entire communities.

Another example: search would bevastly better: you could search conversationally by typing in everyday naturallanguage and you would get precisely what you asked for, or even what youneeded but didn’t know how to ask for correctly, and nothing else. Your searchengine could even ask you questions to help you narrow what you want. You wouldfinally be able to converse with software in ordinary speech and it would understandyou.

The process of discovery would be easier too. You could have software agent that worked as your personal recommendation agent. It would constantly be looking in all the places you read or participate in for things that are relevant to your past, present and potential future interests and needs. It could then alert you in a contextually sensitive way, knowing how to reach you and how urgently to mark things. As you gave it feedback it could learn and do a better job over time.

Going even further with this,semantically-aware software – software that is aware of context, software thatunderstands knowledge – isn’t just for helping you with your information, itcan also help to enrich and facilitate, and even partially automate, yourcommunication and commerce (when you want it to). So for example, your software could help you with your email. It would be able to recommend responses to messages for you, or automate the process. It would be able to enrich your messaging anddiscussions by automatically cross-linking what you are speaking about withrelated messages, discussions, documents, Web sites, subject categories,people, organizations, places, events, etc.

Shopping and marketplaces wouldalso become better – you could search precisely for any kind of product, withany specific attributes, and find it anywhere on the Web, in any store. You could post classified ads and automatically get relevant matches according to your priorities, from all over the Web, or only from specific places and parties that match your criteria for who you trust. You could also easily invent a new custom datastructure for posting classified ads for a new kind of product or service and publishit to the Web in a format that other Web services and applications couldimmediately mine and index without having to necessarily integrate with yoursoftware or data schema directly.

You could publish an entiredatabase to the Web and other applications and services could immediately startto integrate your data with their data, without having to migrate your schemaor their own. You could merge data from different data sources together to create new data sources without having to ever touch or look at an actual database schema.

Bumps on the Road

The above examples illustrate thepotential of the Semantic Web today, but the reality on the ground is that the technology isstill in the early phases of evolution. Even for experienced software engineersand Web developers, it is difficult to apply in practice. The main obstaclesare twofold:

(1) The Tools Problem:

There are very few commercial-gradetools for doing anything with the Semantic Web today – Most of the tools forbuilding semantically-aware applications, or for adding semantics toinformation are still in the research phase and were designed for expertcomputer scientists who specialize in knowledge representation, artificialintelligence, and machine learning.

These tools require a largelearning curve to work with and they don’t generally support large-scaleapplications – they were designed mainly to test theories and frameworks, notto actually apply them. But if the Semantic Web is ever going to becomemainstream, it has to be made easier to apply – it has to be made moreproductive and accessible for ordinary software and content developers.

Fortunately, the tools problem isalready on the verge of being solved. Companies such as my own venture, RadarNetworks, are developing the next generation of tools for building Semantic Webapplications and Semantic Web sites. These tools will hide most of thecomplexity, enabling ordinary mortals to build applications and content thatleverage the power of semantics without needing PhD’s in knowledge representation.

(2) The Ontology Problem:

The Semantic Web providesframeworks for defining systems of formally defined concepts called “ontologies,”that can then be used to connect information to context in an unambiguous way. Withoutontologies, there really can be no semantics. The ontologies ARE the semantics,they define the meanings that are so essential for connecting information tocontext.

But there are still few widely used or standardized ontologies. Andgetting people to agree on common ontologies is not generally easy. Everyonehas their own way of describing things, their own worldview, and let’s face itnobody wants to use somebody else’s worldview instead of their own.Furthermore, the world is very complex and to adequately describe all the knowledgethat comprises what is thought of as “common sense” would require a very largeontology (and in fact, such an ontology exists – it’s called Cyc and it is solarge and complex that only experts can really use it today).

Even to describe the knowledge ofjust a single vertical domain, such as medicine, is extremely challenging. Tomake matters worse, the tools for authoring ontologies are still very hard touse – one has to understand the OWL language and difficult, buggy ontologyauthoring tools in order to use them. Domain experts who are non-technical andnot trained in formal reasoning or knowledge representation may find theprocess of designing ontologies frustrating using current tools. What is needed are commercial quality tools for buildingontologies that hide the underlying complexity so that people can just pourtheir knowledge into them as easily as they speak. That’s still a ways off, butnot far off. Perhaps ten years at the most.

Of course the difficulty ofdefining ontologies would be irrelevant if the necessary ontologies alreadyexisted. Perhaps experts could define them and then everyone else could justuse them? There are numerous ontologies already in existence, both on thegeneral level as well as about specific verticals. However in my own opinion,having looked at many of them, I still haven’t found one that has the rightbalance of coverage of the necessary concepts most applications need, andaccessibility and ease-of-use by non-experts. That kind of balance is arequirement for any ontology to really go mainstream.

Furthermore, regarding the presentcrop of ontologies, what is still lacking is standardization. Ontologists havenot agreed on which ontologies to use. As a result it’s anybody’s guess whichontology to use when writing a semantic application and thus there is a highdegree of ontology diversity today. Diversity is good, but too much diversityis chaos.

Applications that use differentontologies about the same things don’t automatically interoperate unless theirontologies have been integrated. This is similar to the problem of databaseintegration in the enterprise. In order to interoperate, different applicationsthat use different data schemas for records about the same things, have to bemapped to each other somehow – either at the application-level or the data-level.This mapping can be direct or through some form of middleware.

Ontologies canbe used as a form of semantic middleware, enabling applications to be mapped atthe data-level instead of the applications-level. Ontologies can also be usedto map applications at the applications level, by making ontologies of Webservices and capabilities, by the way. This is an area in which a lot ofresearch is presently taking place.

The OWL language can expressmappings between concepts in different ontologies. But if there are manyontologies, and many of them partially overlap, it is a non-trivial task toactually make the mappings between their concepts.

Even though concept A inontology one and concept B in ontology two may have the same names, and evensome of the same properties, in the context of the rest of the concepts intheir respective ontologies they may imply very different meanings. So simplymapping them as equivalent on the basis of their names is not adequate, theirconnections to all the other concepts in their respective ontologies have to beconsidered as well. It quickly becomes complex. There are some potential waysto automate the construction of mappings between ontologies however – but theyare still experimental. Today, integrating ontologies requires the help ofexpert ontologists, and to be honest, I’m not sure even the experts have itfigured out. It’s more of an art than a science at this point.

Darwinian Selection of Ontologies

All that is needed for mainstream adoption to begin is for a largebody of mainstream content to become semantically tagged andaccessible. This will cause whatever ontology is behind that content to become popular.

When developers see that there is significant content andtraction around aparticular ontology, they will use that ontology for their ownapplicationsabout similar concepts, or at least they will do the work of mappingtheir ownontology to it, and in this way the world will converge in a Darwinianfashionaround a few main ontologies over time.

These main ontologies will then beworth thetime and effort necessary to integrate them on a semantic level,resulting in acohesive Semantic Web. We may in fact see Darwinian natural selection take place not just at the ontology level, but at the level of pieces of ontologies.

A certain ontology may do a good job of defining what a person is, while another may do a good job of defining what a company is. These definitions may be used for a lot of content, and gradually they will become common parts of an emergent meta-ontology comprised of the most-popular pieces from thousands of ontologies. This could be great or it could be a total mess. Nobody knows yet. It’s a subject for further research.

Making Sense of Ontologies

Since ontologies are so important,it is helpful to actually understand what an ontology really is, and what itlooks like. An ontology is a system of formally defined related concepts. Forexample, a simple ontology is this set of statements such as this:

A human is a living thing.

A person is a human.

A person may have a first name.

A person may have a last name.

A person must have one and only onedate of birth.

A person must have a gender.

A person may be socially related toanother person.

A friendship is a kind of socialrelationship.

A romantic relationship is a kindof friendship.

A marriage is a kind of romanticrelationship.

A person may be in a marriage withonly one other person at a time.

A person may be employed by anemployer.

An employer may be a person or anorganization.

An organization is a group ofpeople.

An organization may have a productor a service.

A company is a type organization.

We’ve just built a simple ontologyabout a few concepts: humans, living things, persons, names, socialrelationships, marriages, employment, employers, organizations, groups,products and services. Within this system of concepts there is particular logic,some constraints, and some structure. It may or may not correspond to yourworldview, but it is a worldview that is unambiguously defined, can becommunicated, and is internally logically consistent, and that is what isimportant.

The Semantic Web approach providesan open-standard language, OWL, for defining ontologies. OWL also provides fora way to define instances of ontologies. Instances are assertions within theworldview that a given ontology provides. In other words OWL provides a meansto make statements that connect information to the ontology so that softwarecan understand its meaning unambiguously. For example, below is a set ofstatements based on the above ontology:

There exists a person x.

Person x has a first name “Sue”

Person x has a last name “Smith”

Person x has a full name “Sue Smith”

Sue Smith was born on June 1, 2005

Sue Smith has a gender: female

Sue Smith has a friend: Jane, who isanother person.

Sue Smith is married to: Bob, anotherperson.

Sue Smith is employed by Acme, Inc, a company

Acme Inc. has a product, Widget2.0.

The set of statements above, plusthe ontology they are connected to, collectively comprise a knowledge basethat, if represented formally in the OWL markup language, could be understoodby any application that speaks OWL in the precise manner that it was intendedto be understood.

Making Metadata

The OWL language provides a way tomarkup any information such as a data record, an email message or a Web pagewith metadata in the form of statements that link particular words or phrasesto concepts in the ontology. When software applications that understand OWLencounter the information they can then reference the ontology and figure outexactly what the information means – or at least what the ontology says that itmeans.

But something has to add thesesemantic metadata statements to the information – and if it doesn’t add them or adds thewrong ones, then software applications that look at the information will getthe wrong idea. And this is another challenge – how will all this metadata getcreated and added into content? People certainly aren’t going to add it all byhand!

Fortunately there are many ways tomake this easier. The best approach is to automate it using special softwarethat goes through information, analyzes the meaning and adds semantic metadataautomatically. This works today, but the software has to be trained or providedwith rules and that takes some time. It also doesn’t scale cost-effectively tovast data-sets.

Alternatively, individuals can beprovided with ways to add semantics themselves as they author information. Whenyou post your resume in a semantically-aware job board, you could fill out aform about each of your past jobs, and the job board would connect that data toappropriate semantic concepts in an underlying employment ontology. As anend-user you would just fill out a form like you are used to doing;under-the-hood the job board would add the semantics for you.

Another approach is to leveragecommunities to get the semantics. We already see communities that are addingbasic metadata “tags” to photos, news articles and maps. Already a few simpletypes of tags are being used pseudo-semantically: subject tags and geographicaltags. These are primitive forms of semantic metadata. Although they are notexpressed in OWL or connected to formal ontologies, they are at leastsemantically typed with prefixes or by being entered into fields or specificnamespaces that define their types.

Tagging by Example

There may also be another solution to the problem of how to add semantics to content in the not to distant future. Once asuitable amount of content has been marked up with semantic metadata,it may be possible, through purely statistical forms of machinelearning, for software to begin to learn how to do a pretty good job ofmarking up new content with semantic metadata.

For example, if thestring “Nova Spivack” is often marked up with semantic metadata statingthat it indicates a person, and not just any person but a specificperson that is abstractly represented in a knowledge base somewhere,then when software applications encounter a new non-semanticallyenhanced document containing strings such as “Nova Spivack” or”Spivack, Nova” they can make a reasonably good guess that thisindicates that same specific person, and they can add the necessarysemantic metadata to that effect automatically.

As more and more semanticmetadata is added to the Web and made accessible it constitutes a statisticaltraining set that can be learned and generalized from. Although humansmay need to jump-start the process with some manually semantic tagging,it might not be long before software could assist them and eventuallydo all the tagging for them. Only in special cases would software needto ask a human for assistance — for example when totally new terms orexpressions were encountered for the first several times.

The technology for doing this learning already exists — and actually it’s not very different from how search engines like Google measure the community sentiment around web pages. Each time something is semantically tagged with a certain meaning that constitutes a “vote” for it having that meaning. The meaning that gets the most votes wins. It’s an elegant, Darwinian, emergent approach to learning how to automatically tag the Web.

One this is certain, if communities were able to tagthings with more types of tags, and these tags were connected to ontologies andknowledge bases, that would result in a lot of semantic metadata being added tocontent in a completely bottom-up, grassroots manner, and this in turn would enable this process to start to become automated or at least machine-augmented.

Getting the Process Started

But making the userexperience of semantic tagging easy (and immediately beneficial) enough that regular people will do it, is a challenge that has yet to be solved.However, it will be solved shortly. It has to be. And many companies andresearchers know this and are working on it right now. This does have to be solved to get the process of jump-starting the Semantic Web started.

I believe that the Tools Problem – the lack of commercial grade tools forbuilding semantic applications – is essentially solved already (although theproducts have not hit the market yet; they will within a few years at most).The Ontology Problem is further from being solved. I think the way this problemwill be solved is through a few “killer apps” that result in the building up ofa large amount of content around particular ontologies within particular onlineservices.

Where might we see this content initially arising? In my opinion it will most likely be within vertical communities of interest, communities of practice, and communities of purpose. Within such communities there is a need to create a common body of knowledge and to make that knowledge more accessible, connected and useful.

The Semantic Web can really improve the quality of knowledge and user-experience within these domains. Because they are communities, not just static content services, these organizations are driven by user-contributed content — users play a key role in building content and tagging it. We already see this process starting to take place in communities such as Flickr, del.icio.us, the Wikipedia and Digg. We know that communities of people do tag content, and consume tagged content, if it is easy and beneficial enough for to them to do so.

In the near future we may see miniature Semantic Webs arising around particular places, topics and subject areas, projects, and other organizations. Or perhaps, like almost every form of new media in recent times, we may see early adoption of the Semantic Web around online porn — what might be called “the sementic web.”

Whether you like it or not, it is a fact that pornography was one of the biggest drivers of early mainstream adoption of personal video technology, CD-ROMs, and also of the Internet and the Web.

But I think it probably is not necessary this time around. While, I’m sure that the so-called “sementic web” could become better from the Semantic Web, it isn’t going to be the primary driver of adoption of the Semantic Web. That’s probably a good thing — the world can just skip over that phase of development and benefit from this technology with both hands so to speak.

The World Wide Database

In some ways one could think of theSemantic Web as “the world wide database” – it does for the meaning of data records what theWeb did for the formatting documents. But that’s just the beginning. It actually turnsdocuments into richer data records. It turns unstructured data into structureddata. All data becomes structured data in fact. The structure is not merelydefined structurally, but it is defined semantically.

In other words, it’s notmerely that for example, a data record or document can be defined in such a wayas to specify that it contains a certain field of data with a certain label ata certain location – it defines what that field of data actually means in anunambiguous, machine understandable way. If all you want is a Web of data,XML is good enough. But if you want to make that data interoperable and machineunderstandable then you need RDF and OWL – the Semantic Web.

Like any database,the Semantic Web, or rather the myriad mini-semantic-webs that will comprise it,have to overcome the challenge of data integration. Ontologies provide a betterway to describe and map data, but the data still has to be described andmapped, and this does take some work. It’s not a magic bullet.

The Semantic Webmakes it easier to integrate data, but it doesn’t completely remove the dataintegration problem altogether. I think the eventual solution to this problemwill combine technology and community folksonomy oriented approaches.

The Semantic Web in HistoricalContext

Let’s transition now and zoom out to see the bigger picture. The Semantic Webprovides technologies for representing and sharing knowledge in new ways. Inparticular, it makes knowledge more accessible to software, and thus to otherpeople. Another way of saying this is that it liberates knowledge fromparticular human minds and organizations – it provides a way to make knowledgeexplicit, in a standardized format that any application can understand. This isquite significant. Let’s put this in historical perspective.

Before the invention of the printing press, there were two ways to spreadknowledge – one was orally, the other was in some symbolic form such as art orwritten manuscripts. The oral transmission of knowledge had limited range and ahigh error-rate, and the only way to learn something was to meet someone whoknew it and get them to tell you. The other option, symbolic communicationthrough art and writing, provided a means to communicate knowledgeindependently of particular people – but it was only feasible to produce a fewcopies of any given artwork or manuscript because they had to be copied byhand. So the transmission of knowledge was limited to small groups or at leastsmall audiences. Basically, the only way to get access to this knowledge was tobe one of the lucky few who could acquire one of its rare physical copies.

The invention of the printing press changed this – for the first timeknowledge could be rapidly and cost-effectively mass-produced and mass-distributed.Printing made it possible to share knowledge with ever-larger audiences. Thisenabled a huge transformation for human knowledge, society, government,technology – really every area of human life was transformed by thisinnovation.

The World Wide Web made the replication and distribution of knowledge eveneasier – With the Web you don’t even have to physically print or distributeknowledge anymore, the cost of distribution is effectively zero, and everyonehas instant access to everything from anywhere, anytime. That’s a lot betterthan having to lug around a stack of physical books. Everyone potentially haswhatever knowledge they need with no physical barriers. This has been anotherhuge transformation for humanity – and it has affected every area of humanlife. Like the printing press, the Web fundamentally changed the economics ofknowledge.

The Semantic Web is the next big step in this process – it will make all theknowledge of the human race accessible to software. For the first time,non-human things (software applications) will be able to start working withhuman knowledge to do things (for humans) on their own. This is a big leap – aleap like the emergence of a new species, or the symbiosis of two existingspecies into a new form of life.

The printing press and the Web changed the economics of replicating,distributing and accessing knowledge. The Semantic Web changes the economics ofprocessing knowledge. Unlike the printing press and the Web, the Semantic Webenables knowledge to be processed by non-human things.

In other words, humans don’t have to do all the thinking on their own, theycan be assisted by software. Of course we humans have to at least first createthe software (until we someday learn to create software that is smart enough tocreate software too), and we have to create the ontologies necessary for thesoftware to actually understand anything (until we learn to create software thatis smart enough to create ontologies too), and we have to add the semanticmetadata to our content in various ways (until our software is smart enough todo this for us, which it almost is already). But once we do the initial work ofmaking the ontologies and software, and adding semantic metadata, the systemstarts to pick up speed on its own, and over time the amount of work we humanshave to do to make it all function decreases. Eventually, once the system hasencoded enough knowledge and intelligence, it starts to function withoutneeding much help, and when it does need our help, it will simply ask us andlearn from our answers.

This may sound like science-fiction today, but in fact it a lot of this isalready built and working in the lab. The big hurdle is figuring out how to getthis technology to mass-market. That is probably as hard as inventing thetechnology in the first place. But I’m confident that someone will solve iteventually.

Once this happens the economics of processing knowledge will truly bedifferent than it is today. Instead of needing an actual real-live expert, theknowledge of that expert will be accessible to software that can act as theirproxy – and anyone will be able to access this virtual expert, anywhere,anytime. It will be like the Web – but instead of just information beingaccessible, the combined knowledge and expertise of all of humanity will alsobe accessible, and not just to people but also to software applications.

The Question of Consciousness

The Semantic Web literally enables humans to share their knowledge with eachother and with machines. It enables the virtualization of human knowledge andintelligence. With respect to machines, in doing this, it will lend machines“minds” in a certain sense – namely in that they will at least be able tocorrectly interpret the meaning of information and replicate the expertise ofexperts.

But will these machine-minds be conscious? Will they be aware of themeanings they interpret, or will they just be automatons that are simplyfollowing instructions without any awareness of the meanings they areprocessing? I doubt that software will ever be conscious, because from what Ican tell consciousness — or what might be called the sentient awareness ofawareness itself as well as other things that are sensed — is an immaterialphenomena that is as fundamental as space, time and energy — or perhaps evenmore fundamental. But this is just my personal opinion after having searchedfor consciousness through every means possible for decades. It just cannot befound to be something, yet it is definitely and undeniably taking place.

Consciousness can be exemplified through the analogy of space (but unlikespace, consciousness has this property of being aware, it’s not a mere lifelessvoid). We all agree space is there, but nobody can actually point to itsomewhere, and nobody can synthesize space. Space is immaterial andfundamental. It is primordial. So is electricity. Nobody really knows whatelectricity is ultimately, but if you build the right kind of circuit you canchannel it and we’ve learned a lot about how to do that.

Perhaps we may figure out how to channel consciousness like we channelelectricity with some sort of synthetic device someday, but I think that ishighly unlikely. I think if you really want to create consciousness it’s mucheasier and more effective to just have children. That’s something ordinarymortals can do today with the technology they were born with. Of course whenyou have children you don’t really “create” their consciousness, it seems to bethere on its own. We don’t really know what it is or where it comes from, orwhen it arises there. We know very little about consciousness today.Considering that it is the most fundamental human experience of all, it isactually surprising how little we know about it!

In any case, until we truly delve far more deeply into the nature of themind, consciousness will be barely understood or recognized, let aloneexplained or synthesized by anyone. In many eastern civilizations there aremulti-thousand year traditions that focus quite precisely on the nature ofconsciousness. The major religions have all universally concluded thatconsciousness is beyond the reach of science, beyond the reach of concepts,beyond the mind entirely. All those smart people analyzing consciousness for solong, and with such precision, and so many methods of inquiry, may have a pointworth listening to.

Whether or not machines will ever actually “know” or be capable of beingconscious of that meaning or expertise is a big debate, but at least we can allagree that they will be able to interpret the meaning of information and rulesif given the right instructions. Without having to be conscious, software willbe able to process semantics quite well — this has already been proven. It’sworking today.

While consciousness is and may always be a mystery that we cannot synthesize– the ability for software to follow instructions is an established fact. Inits most reduced form, the Semantic Web just makes it possible to providericher kinds of instructions. There’s no magic to it. Just a lot of details. Infact, to play on a famous line, “it’s semantics all the way down.”

The Semantic Web does not require that we make conscious software. It justprovides a way to make slightly more intelligent software. There’s a bigdifference. Intelligence is simply a form of information processing, for themost part. It does not require consciousness — the actual awareness of what isgoing on — which is something else altogether.

While highly intelligentsoftware may need to sense its environment and its own internal state andreason about these, it does not actually have to be conscious to do this. Theseoperations are for the most part simple procedures applied vast numbers of timeand in complex patterns. Nowhere in them is there any consciousness nor doesconsciousness suddenly emerge when suitable levels of complexity are reached.

Consciousness is something quite special and mysterious. And fortunately forhumans, it is not necessary for the creation of more intelligent software, noris it a byproduct of the creation of more intelligent software, in my opinion.

The Intelligence of the Web

So the real point of the Semantic Web is that it enables the Web to becomemore intelligent. At first this may seem like a rather outlandish statement,but in fact the Web is already becoming intelligent, even without the SemanticWeb.

Although the intelligence of the Web is not very evident at first glance,nonetheless it can be found if you look for it. This intelligence doesn’t existacross the entire Web yet, it only exists in islands that are few and farbetween compared to the vast amount of information on the Web as a whole. Butthese islands are growing, and more are appearing every year, and they arestarting to connect together. And as this happens the collective intelligenceof the Web is increasing.

Perhaps the premier example of an “island of intelligence” is theWikipedia, but there are many others: The Open Directory, portals such as Yahooand Google, vertical content providers such as CNET and WebMD, commercecommunities such as Craigslist and Amazon, content oriented communities such asLiveJournal, Slashdot, Flickr and Digg and of course the millions of discussionboards scattered around the Web, and social communities such as MySpace andFacebook. There are also large numbers of private islands of intelligence onthe Web within enterprises — for example the many online knowledge andcollaboration portals that exist within businesses, non-profits, andgovernments.

What makes these islands “intelligent” is that they are places where people(and sometimes applications as well) are able to interact with each other tohelp grow and evolve collections of knowledge. When you look at them close-upthey appear to be just like any other Web site, but when you look at what theyare doing as a whole – these services are thinking.They are learning, self-organizing, sensing their environments, interpreting,reasoning, understanding, introspecting, and building knowledge. These are theactivities of minds, of intelligent systems.

The intelligence of a system such as the Wikipedia exists on several levels– the individuals who author and edit it are intelligent, the groups that helpto manage it are intelligent, and the community as a whole – which isconstantly growing, changing, and learning – is intelligent.

Flickr and Digg also exhibit intelligence. Flickr’s growing system of tagsis the beginnings of something resembling a collective visual sense organ onthe Web. Images are perceived, stored, interpreted, and connected to conceptsand other images. This is what the human visual system does. Similarly, Digg isa community that collectively detects, focuses attention on, and interpretscurrent news. It’s not unlike a primitive collective analogue to the humanfacility for situational awareness.

There are many other examples of collective intelligence emerging on theWeb. The Semantic Web will add one more form of intelligent actor to the mix –intelligent applications. In the future, after the Wikipedia is connected tothe Semantic Web, as well as humans, it will be authored and edited by smartapplications that constantly look for new information, new connections, and newinferences to add to it.

Although the knowledge on the Web today is still mostly organized withindifferent islands of intelligence, these islands are starting to reach out andconnect together. They are forming trade-routes, connecting their economies,and learning each other’s languages and cultures. The next-step will be forthese islands of knowledge to begin to share not just content and services, butalso their knowledge — what they know about their content and services. The SemanticWeb will make this possible, by providing an open format for the representationand exchange of knowledge and expertise.

When applications integrate their content using the Semantic Web they willalso be able to integrate their context, their knowledge – this will make thecontent much more useful and the integration much deeper. For example, when anapplication imports photos from another application it will also be able toimport semantic metadata about the meaning and connections of those photos.Everything that the community and application know about the photos in theservice that provides the content (the photos) can be shared with the servicethat receives the content. Better yet, there will be no need for customapplication integration in order for this to happen: as long as both servicesconform to the open standards of the Semantic Web the knowledge is instantlyportable and reusable.

Freeing Intelligence from Silos

Today much of the real value of the Web (and in the world) is still lockedaway in the minds of individuals, the cultures of groups and organizations, andapplication-specific data-silos. The emerging Semantic Web will begin to unlockthe intelligence in these silos by making the knowledge and expertise theyrepresent more accessible and understandable.

It will free knowledge and expertise from the narrow confines of individualminds, groups and organizations, and applications, and make them not only moreinteroperable, but more portable. It will be possible for example for a personor an application to share everything they know about a subject of interest aseasily as we share documents today. In essence the Semantic Web provides acommon language (or at least a common set of languages) for sharing knowledgeand intelligence as easily as we share content today.

The Semantic Web also provides standards for searching and reasoning moreintelligently. The SPARQL query language enables any application to ask forknowledge from any other application that speaks SPARQL. Instead of merekeyword search, this enables semantic search. Applications can search forspecific types of things that have particular attributes and relationships toother things.

In addition, standards such as SWRL provide formalisms for representing andsharing axioms, or rules, as well. Rules are a particular kind of knowledge –and there is a lot of it to represent and share, for example proceduralknowledge, and logical structures about the world. An ontology provides a meansto describe the basic entities, their attributes and relations, but rulesenable you to also make logical assertions and inferences about them. Withoutgoing into a lot of detail about rules and how they work here, the importantpoint to realize is that they are also included in the framework. All forms ofknowledge can be represented by the Semantic Web.

Zooming Way, Waaaay Out

So far in this article, I’ve spenta lot of time talking about plumbing – the pipes, fluids, valves, fixtures,specifications and tools of the Semantic Web. I’ve also spent some time onillustrations of how it might be useful in the very near future to individuals,groups and organizations. But where is it heading after this? What is thelong-term potential of this and what might it mean for the human race on ahistorical time-scale?

For those of you who would prefer not to speculate, stop reading here. Forthe rest of you, I believe that the true significance of the Semantic Web, on along-term timescale is that it provides an infrastructure that will enable theevolution of increasingly sophisticated forms of collective intelligence. Ultimatelythis will result in the Web itself becoming more and more intelligent, untilone day the entire human species together with all of its software andknowledge will function as something like a single worldwide distributed mind –a global mind.

Just the like the mind of a single human individual, the global mind will bevery chaotic, yet out of that chaos will emerge cohesive patterns of thoughtand decision. Just like in an individual human mind, there will be feedbackbetween different levels of order – from individuals to groups to systems ofgroups and back down from systems of groups to groups to individuals. Becauseof these feedback loops the system will adapt to its environment, and to itsown internal state.

The coming global mind will collectively exhibit forms of cognition andbehavior that are the signs of higher-forms of intelligence. It will form andreact to concepts about its “self” – just like an individual human mind. Itwill learn and introspect and explore the universe. The thoughts it thinks maysometimes be too big for any one person to understand or even recognize them –they will be comprised of shifting patterns of millions of pieces of knowledge.

The Role of Humanity

Every person on the Internet will be a part of the global mind. Andcollectively they will function as its consciousness. I do not believe some newform of consciousness will suddenly emerge when the Web passes some thresholdof complexity. I believe that humanity IS the consciousness of the Web anduntil and unless we ever find a way to connect other lifeforms to the Web, orwe build conscious machines, humans will be the only form of consciousness ofthe Web.

When I say that humans will function as the consciousness of the Web I meanthat we will be the things in the system that know. The knowledge of theSemantic Web is what is known, but what knows that knowledge has to besomething other than knowledge. A thought is knowledge, but what knows thatthought is not knowledge, it is consciousness, whatever that is. We can figureout how to enable machines to represent and use knowledge, but we don’t knowhow to make them conscious, and we don’t have to. Because we are alreadyconscious.

As we’ve discussed earlier in this article, we don’t need conscious machines, we just need more intelligent machines.Intelligence – at least basic forms of it – does not require consciousness. It may be the case that the very highest forms of intelligence require or are capable of consciousness. This may mean that software will never achieve the highest levels of intelligence and probably guaranteesthat humans (and other conscious things) will always play a special role in theworld; a role that no computer system will be able to compete with. We providethe consciousness to the system. There may be all sorts of other intelligent,non-conscious software applications and communities on the Web; in fact therealready are, with varying degrees of intelligence. But individual humans, andgroups of humans, will be the only consciousness on the Web.

The Collective Self

Although the software of the Semantic Web will not be conscious we can say that system as a whole contains or is conscious to the extent that human consciousnesses are part of it. And like most conscious entities, it may also start to be self-conscious.

If the Web ever becomes a global mind as I am predicting, will it have a“self?” Will there be a part of the Web that functions as its central self-representation?Perhaps someone will build something like that someday, or perhaps it will evolve.Perhaps it will function by collecting reports from applications and people inreal-time – a giant collective zeitgeist.

In the early days of the Web portals such as Yahoo! provided this function — they were almost real-time maps of the Web and what was happening. Today making such a map is nearly impossible, but services such as Google Zeitgeist at least attempt to provide approximations of it. Perhaps through random sampling it can be done on a broader scale.

My guess is that the global mind will need a self-representation at somepoint. All forms of higher intelligence seem to have one. It’s necessary forunderstanding, learning and planning. It may evolve at first as a bunch ofcompeting self-representations within particular services or subsystems withinthe collective. Eventually they will converge or at least narrow down to just afew major perspectives. There may also be millions of minor perspectives thatcan be drilled down into for particular viewpoints from these top-level “portals.”

The collective self, will function much like the individual self – as amirror of sorts. Its function is simply to reflect. As soon as it exists theentire system will make a shift to a greater form of intelligence – because forthe first time it will be able to see itself, to measure itself, as a whole. Itis at this phase transition when the first truly global collective self-mirroring function evolves, that we can say that the transition from a bunch of cooperating intelligent parts toa new intelligent whole in its own right has taken place.

I think that the collective self, even if it converges on a few majorperspectives that group and summarize millions of minor perspectives, will becommunity-driven and highly decentralized. At least I hope so – because theself-concept is the most important part of any mind and it should be designedin a way that protects it from being manipulated for nefarious ends. At least Ihope that is how it is designed.

Programming the Global Mind

On the other hand, there are times when a little bit of adjustment or guidance iswarranted – just as in the case of an individual mind, the collective selfdoesn’t merely reflect, it effectively guides the interpretation of the pastand present, and planning for the future.

One way to change the direction ofthe collective mind, is to change what is appearing in the mirror of thecollective self. This is a form of programming on a vast scale – When thisprogramming is dishonest or used for negative purposes it is called “propaganda,” but there are cases whereit can be done for beneficial purposes as well. An example of this today ispublic service advertising and educational public television programming. Allforms of mass-media today are in fact collective social programming. When yourealize this it is not surprising that our present culture is violent andmessed up – just look at our mass-media!

In terms of the global mind, ideally one would hope that it would be able tolearn and improve over time. One would hope that it would not have the collective equivalent of psycho-social disorders. To facilitate this, just like any form of higherintelligence, it may need to be taught, and even parented a bit. It also mayneed a form of therapy now and then. These functions could be provided by thepeople who participate in it. Again, I believe that humans serve a vital and irreplaceablerole in this process.

How It All Might Unfold

Now how is this all going to unfold? I believe that there are a number ofkey evolutionary steps that Semantic Web will go through as the Web evolvestowards a true global mind:

1. Representing individual knowledge. The first step is to make individuals’knowledge accessible to themselves. As individuals become inundated withincreasing amounts of information, they will need better ways of managing it,keeping track of it, and re-using it. They will (or already do) need”personal knowledge management.”

2. Connecting individual knowledge. Next, once individual knowledge isrepresented, it becomes possible to start connecting it and sharing it acrossindividuals. This stage could be called “interpersonal knowledgemanagement.”

3. Representing group knowledge. Groups of individuals also need ways ofcollectively representing their knowledge, making sense of it, and growing itover time. Wikis and community portals are just the beginning. The Semantic Webwill take these “group minds” to the next level — it will make the collective knowledge ofgroups far richer and more re-usable.

4. Connecting group knowledge. This step is analogous to connectingindividual knowledge. Here, groups become able to connect their knowledge togetherto form larger collectives, and it becomes possible to more easily access andshare knowledge between different groups in very different areas of interest.

5. Representing the knowledge of the entire Web. This stage — what might becalled “the global mind” — is still in the distant future, but atthis point in the future we will begin to be able to view, search, and navigatethe knowledge of the entire Web as a whole. The distinction here is thatinstead of a collection of interoperating but separate intelligentapplications, individuals and groups, the entire Web itself will begin tofunction as one cohesive intelligent system. The crucial step that enables thisto happen is the formation of a collective self-representation. This enablesthe system to see itself as a whole for the first time.

How it May be Organized

I believe the global mind will be organized mainly in the form of bottom-up and lateral, distributed emergent computation andcommunity — but it will be facilitated by certain key top-down services thathelp to organize and make sense of it as a whole. I think this future Web willbe highly distributed, but will have certain large services within it as well– much like the human brain itself, which is organized into functionalsub-systems for processes like vision, hearing, language, planning, memory,learning, etc.

As the Web gets more complex there will come a day when nobody understandsit anymore – after that point we will probably learn more about how the Web isorganized by learning about the human mind and brain – they will be quitesimilar in my opinion. Likewise we will probably learn a tremendous amountabout the functioning of the human brain and mind by observing how the Webfunctions, grows and evolves over time, because they really are quite similarin at least an abstract sense.

The internet and its software and content is like a brain, and the state ofits software and the content is like its mind. The people on the Internet arelike its consciousness. Although these are just analogies, they are actuallyuseful, at least in helping us to envision and understand this complex system. Asthe field of general systems theory has shown us in the past, systems at verydifferent levels of scale tend to share the same basic characteristics and obeythe same basic laws of behavior. Not only that, but evolution tends to convergeon similar solutions for similar problems. So these analogies may be more thanjust rough approximations, they may be quite accurate in fact.

The future global brain will require tremendous computing and storageresources — far beyond even what Google provides today. Fortunately as Moore’s Law advances thecost of computing and storage will eventually be low enough to do thiscost-effectively. However even with much cheaper and more powerful computingresources it will still have to be a distributed system. I doubt that therewill be any central node because quite simply no central solution will be ableto keep up with all the distributed change taking place. Highly distributed problemsrequire distributed solutions and that is probably what will eventually emergeon the future Web.

Someday perhaps it will be more like a peer-to-peer network, comprised ofapplications and people who function sort of like the neurons in the human brain.Perhaps they will be connected and organized by higher-level super-peers orsuper-nodes which bring things together, make sense of what is going on andcoordinate mass collective activities. But even these higher-level serviceswill probably have to be highly distributed as well. It really will bedifficult to draw boundaries between parts of this system, they will all beconnected as an integral whole.

In fact it may look very much like a grid computing architecture – in whichall the services are dynamically distributed across all the nodes such that atany one time any node might be working on a variety of tasks for differentservices. My guess is that because this is the simplest, most fault-tolerant,and most efficient way to do mass computation, it is probably what will evolvehere on Earth.

The Ecology of Mind

Where we are today in this evolutionary process is perhaps equivalent to therise of early forms of hominids. Perhaps Austrolapithecus or Cro-Magnon, ormaybe the first Homo Sapiens. Compared to early man, the global mind is like the rise of 21^stcentury mega-cities. A lot of evolution has to happen to get there. But itprobably will happen, unless humanity self-destructs first,which I sincerely hope we somehow manage to avoid. And this brings me to afinal point. This vision of the future global mind is highly technological;however I don’t think we’ll ever accomplish it without a new focus on ecology.

Ecology probably conjures up images of hippies and biologists, or maybehippies who are biologists, or at least organic farmers, for most people, but infact it is really the science of living systems and how they work. And anysystem that includes living things is a living system. This means that the Webis a living system and the global mind will be a living system too. As a living system, the Web is an ecosystem and is alsoconnected to other ecosystems. In short, ecology is absolutely essential tomaking sense of the Web, let alone helping to grow and evolve it.

In many ways the Semantic Web and the collective minds, and the global mind,that it enables, can be seen as an ecosystem of people, applications,information and knowledge. This ecosystem is very complex, much like naturalecosystems in the physical world. An ecosystem isn’t built, it’s grown, andevolved. And similarly the Semantic Web, and the coming global mind, will notreally be built, they will be grown and evolved. The people and organizationsthat end up playing a leading role in this process will be the ones thatunderstand and adapt to the ecology most effectively.

In my opinion ecology is going to be the most important science anddiscipline of the 21^st century – it is the science of healthysystems. What nature teaches us about complex systems can be applied to everykind of system – and especially the systems we are evolving on the Web. Inorder to ever have a hope of evolving a global mind, and all the wonderfullevels of species-level collective intelligence that it will enable, we have tonot destroy the planet before we get there. Ecology is the science that cansave us, not the Semantic Web (although perhaps by improving collectiveintelligence, it can help).

Ecology is essentially the science of community – whether biological,technological or social. And community is a key part of the Semantic Web atevery level: communities of software, communities of people, and communities ofgroups. In the end the global mind is the ultimate human community. It is thereward we get for finally learning how to live together in peace and balancewith our environment.

The Necessity of Sustainability

The point of this discussion of the relevance of ecology to the future ofthe Web, and my vision for the global mind, is that I think that it is clearthat if the global mind ever emerges it will not be in a world that is anythinglike what we might imagine. It won’t be like the Borg in Star Trek, it won’t belike living inside of a machine. Humans won’t be relegated to the roles ofslaves or drones. Robots won’t be doing all the work. The entire world won’t becoated with silicon. We won’t all live in a virtual reality. It won’t be one ofthese technological dystopias.

In fact, I think the global mind can only come to pass in a much greener,more organic, healthier, more balanced and sustainable world. Because it willtake a long time for the global mind to emerge, if humanity doesn’t figure outhow to create that sort of a world, it will wipe itself out sooner or later,but certainly long before the global mind really happens. Not only that, butthe global mind will be smart by definition, and hopefully this intelligencewill extend to helping humanity manage its resources, civilizations andrelationships to the natural environment.

The Smart Environment

The global mind also needs a global body so to speak. It’s not going to bean isolated homunculus floating in a vat of liquid that replaces the physicalworld! It will be a smart environment that ubiquitously integrates with ourphysical world. We won’t have to sit in front of computers or deliberatelylogon to the network to interact with the global mind. It will be everywhere.

The global mind will be physically integrated into furniture, houses,vehicles, devices, artworks, and even the natural environment. It will sensethe state of the world and different ecosystems in real-time and alert humansand applications to emerging threats. It will also be able to allocateresources intelligently to compensate for natural disasters, storms, andenvironmental damage – much in the way that the air traffic control systemsallocates and manages airplane traffic. It won’t do it all on its own, humansand organizations will be a key part of the process.

Someday the global mind may even be physically integrated into our bodiesand brains, even down the level of our DNA. It may in fact learn how to curediseases and improve the design of the human body, extending our lives, sensorycapabilities, and cognitive abilities. We may be able to interact with it bythought alone. At that point it will become indistinguishable from a limitedfrom of omniscience, and everyone may have access to it. Although it will onlyextend to wherever humanity has a presence in the universe, within thatboundary it will know everything there is to know, and everyone will be able toknow any of it they are interested in.

Enabling a Better World

By enabling greater forms of collective intelligence to emerge we really arehelping to make a better world, a world that learns and hopefully understandsitself well enough to find a way to survive. We’re building something thatsomeday will be wonderful – far greater than any of us can imagine. We’re helpingto make the species and the whole planet more intelligent. We’re building thetools for the future of human community. And that future community, if it ever arrives,will be better, more self-aware, more sustainable than the one we live intoday.

I should also mention that knowledge is power, and power can be used forgood or evil. The Semantic Web makes knowledge more accessible. This puts more power in the hands of the many, not just the few. As long as we stick to this vision — we stick to making knowledge open and accessible, using open standards, in as distributed a fashion as we can devise, then the potential power of the Semantic Web will be protected against being coopted or controlled by the few at the expense of the many. This is where technologists really have to be socially responsible when making development decisions. It’s important that we build a more open world, not a less open world. It’s important that we build a world where knowledge, integration and unification are balanced with respect for privacy, individuality, diversity and freedom of opinion.

But I am not particularly worried that the Semantic Web and the future globalmind will be the ultimate evil – I don’t think it is likely that we will end upwith a system of total control dominated by evil masterminds with powerfulSemantic Web computer systems to do their dirty work. Statistically speaking, criminal empires don’t last very long because theyare run by criminals who tend to be very short-sighted and who also surroundthemselves with other criminals who eventually unseat them, or theyself-destruct. It’s possible that the Semantic Web, like any other technology,may be used by the bad guys to spy on citizens, manipulate the world, and doevil things. But only in the short-term.

In the long-term either our civilization will get tired of endlesssuccessions of criminal empires and realize that the only way to actuallysurvive as a species is to invent a form of government that is immune to beingtaken over by evil people and organizations, or it will self-destruct. Eitherway, that is a hurdle we have to cross before the global mind that I envisioncan ever come about. Many civilizations came before ours, and it is likely thatours will not be the last one on this planet. It may in fact be the case that adifferent form of civilization is necessary for the global mind to emerge, andis the natural byproduct of the emergence of the global mind.

We know that the global mind cannot emerge anytime soon, and therefore, ifit ever emerges then by definition it must be in the context of a civilizationthat has learned to become sustainable. A long-term sustainable civilization is a non-evil civilization. And that is why I think it is a safebet to be so optimistic about the long-term future of this trend.

Is Moral Judgement Hard-Wired Into the Brain?

Nova — Thu, 02 Nov 2006 01:33:59 +0000

A Harvard University researcher believes that moral judgement is hard-wired into the brain:

The moral grammar now universal among people presumably evolved to its
final shape during the hunter-gatherer phase of the human past, before
the dispersal from the ancestral homeland in northeast Africa some
50,000 years ago. This may be why events before our eyes carry far
greater moral weight than happenings far away, Dr. Hauser believes,
since in those days one never had to care about people remote from ones
environment.

Dr. Hauser believes that the moral grammar may have evolved through the
evolutionary mechanism known as group selection. A group bound by
altruism toward its members and rigorous discouragement of cheaters
would be more likely to prevail over a less cohesive society, so genes
for moral grammar would become more common.

Is There Room for The Soul? – Good Article on Cognitive Science

Nova — Tue, 17 Oct 2006 20:13:31 +0000

This is a surprisingly good article on the nature of consciousness — providing a survey of the current state-of-the-art in cognitive science research. It covers the question from a number of perspectives and interviews many of the leading current researchers.

Why Machines Will Never be Conscious

Nova — Tue, 17 Oct 2006 17:08:00 +0000

Below is the text of my bet on Long Bets. Go there to vote.

“By 2050 no synthetic computer nor machine intelligence will have become truly self-aware (ie. will become conscious).”

Spivack’s Argument:

(This summary includes my argument, a method for judging the outcomeof this bet and some other thoughts on how to measure awareness…)

A. MY PERSPECTIVE…

Even if a computer passes the Turing Test it will not really beaware that it has passed the Turing Test. Even if a computer seems tobe intelligent and can answer most questions as well as an intelligent,self-aware, human being, it will not really have a continuum ofawareness, it will not really be aware of what it seems to “think” or”know,” it will not have any experience of it’s own reality or being.It will be nothing more than a fancy inanimate object, a clevermachine, it will not be a truly sentient being.

Self-awareness is not the same thing as merely answering questionsintelligently. Therefore even if you ask a computer if it is self-awareand it answers that it is self-aware and that it has passed the TuringTest, it will not really be self-aware or really know that it haspassed the Turing Test.

AsJohn Searle and others have pointed out, the Turing Test does notactually measure awareness, it just measures informationprocessing—particularly the ability to follow rules or at leastimitate a particular style of communication. In particular it measuresthe ability of a computer program to imitate humanlike dialogue, whichis different than measuring awareness itself. Thus even if we succeedin creating good AI, we won’t necessarily succeed in creating AA(“Artificial Awareness”).

But why does this matter? Becauseultimately, real awareness may be necessary to making an AI that is asintelligent as a human sentient being. However, since AA istheoretically impossible in my opinion, truly self-aware AI will neverbe created and thus no AI will ever be as intelligent as a humansentient being even if it manages to fool someone into thinking it is(and thus passing the Turing Test).

In my opinion, awareness isnot an information process at all and will never be simulated orsynthesized by any information process. Awareness cannot be measured byan information processing system, it can only be measured by awarenessitself—something no formal information processing system can eversimulate or synthesize.

One might ask how it is that a humanhas awareness then? My answer is that awareness does not arise from thebody or the brain, nor does it arise from any physical cause. Awarenessis not in the body or the brain, but rather the body and the brain arein awareness. The situation is analagous to a dream, a simulation orvirtual reality, such as that portrayed in the popular film “TheMatrix.”

We exist in the ultimate virtual reality. The mediumof this virtual reality is awareness. That is to say that whateverappears to be happening “out there” or “within the mind” is happeningwithin a unified, nondualistic field of awareness: both the “subject”and the “object” exist equally within this field and neither is thesource of awareness.

This is similar to the case where weproject ourselves as dream protagonists in our own dreams—even thoughour dream bodies appear to be different than other dream-images theyare really equally dream appearances, they are no more fundamental thandream-objects. We identify with our dream-bodies out of habit andbecause it’s practical because the stories that take place appear fromthe perspective of particular bodies. But just because this virtualreality is structured as if awareness is coming from within our heads,it does not mean that is actually the case. In fact, quite the oppositeis taking place.

Awareness is not actually “in” the VR, the VR is”in” awareness. Things are exactly the opposite of how they appear. Ofcourse this is just an analogy—for example, unlike the Matrix, thevirtual reality we live in is not running on some giant computersomewhere and there is no other hidden force controlling it from behindthe scenes. Awareness is the fabric of reality and there is nothingdeeper, nothing creating it, it is not running on some cosmic computer,it comes out of of nowhere yet everything else comes out of it.

Ifwe look for awareness we can’t find anything to grasp, it is empty yetnot a mere nothingness, it is an emptiness that is awake, creative,alert, radiant, self-realizing.

Awareness is empty andfundamental like space, but it goes beyond space for it is also lucid.If we look for space we don’t find anything there. Nobody has evertouched or grasped space directly! But unlike space, awareness can atleast be measured directly–it can measure itself, it knows its ownnature.

Awareness is simply fundamental, a given, theunderlying meta-reality in which everything appears. How did it come tobe? That is unanswerable. What is it? That is unanswerable as well. Butthere is no doubt that awareness is taking place. Each sentient beinghas a direct and intimate experience of their own self-awareness.

Each of us experiences a virtual reality in which we and our world areprojections. That which both projects these projections and experiencesthem is awareness. This is like saying that the VR inherently knows itsown content. But in my opinion this knowing comes from outside thesystem, not from some construct that we can create inside it. So anyawareness that arises comes from the transcendental nature of realityitself, not from our bodies, minds, or any physical system within aparticular reality.

So is there one cosmic awareness out therethat we are all a part of? Not exactly, there is not one awareness norare there many awarenesses because awareness is not a physical thingand cannot be limited by such logical materialist extremes. After allif it is not graspable how can we say it is one or many or any otherlogical combination of one or many? All we can say is that we are it,whatever it is, and that we cannot explain it further. In beingawareness, we are all equal, but we are clearly not the same. We aredifferent projections and on a relative level we are each unique, eventhough on an ultimate level perhaps we are also unified by beingprojections within the same underlying continuum. Yet this continuum isfundamentally empty, impossible to locate or limit, and infinitelybeyond the confines of any formal system or universe, so it cannotreally be called a “thing” and thus we are not “many” or “one” inactuality, what we really are is totally beyond such dualisticdistinctions.

Awareness is like space or reality, something sofundamental, so axiomatic, that it is impossible to prove, grasp ordescribe from “inside” the system using the formal logical tools of thesystem. Since nothing is beyond awareness, there is no outside, no wayto ever gain a perspective on awareness that is not mediated byawareness itself.

Therefore there is no way to reduce awarenessto anything deeper; there is no way to find anything more fundamentalthan awareness. But despite this awareness can be directly experienced,at least by itself.

That which is aware is self-aware.Self-awareness is the very nature of awareness. The self-awareness ofawareness does not come from something else, it is inherent toawareness itself. Only awareness is capable of awareness. Nothing thatis not aware can ever become aware.

This means awareness istruly fundamental, it has always been present everywhere. Awareness isinherent in the universe as the very basis of everything, it is notsomething anyone can synthesize and we cannot build a machine that cansuddenly experience awareness.

Only beings who are awarealready can ever experience awareness. The fact that we are aware nowmeans that we were always aware, even before we were born! Otherwise wenever could have become aware in the first place!

Each of us “is”awareness. The experience of being aware is unique and undeniable. Ithas its own particular nature, but this cannot be expressed it can onlybe known directly. There is no sentient being that is not aware.Furthermore, it would be a logical contradiction to claim that “I amnot aware that I am aware” or “that I am aware that I am not aware” andthus if anyone claims they are not aware or have ever experienced, orcan even imagine, there not being awareness they are lying. There isnobody who does not experience their own awareness, even if they don’trecognize or admit that they experience it.

The experience ofbeing self-aware is the unique experience of “being” — an experienceso basic that it is indescribable in terms of anything else —something that no synthetic computer will ever have.

Eventually, it will be proved that no formal information processingsystem is capable of self-awareness and that thus formal computerscannot be self-aware in principle. This proof will use the abstractself-referential structure of self-awareness to establish that noformal computer can ever be self-aware.

Simplyput, computers and computer programs cannot be truly self-referential:they always must refer to something else—there must at least be a setof fixed meta-rules that are not self-referential for a computer orprogram to work. Awareness is not like this however, awareness isperfectly self-referential without referring to anything else.

Thequestion will then arise as to what self-awareness is and how it ispossible. We will eventually conclude that systems that are self-awareare not formal systems and that awareness must be at least asfundamental as, or more fundamental than, space, time and energy.

Currentlymost scientists and non-scientists consider the physical world to beoutside of awareness and independent of it. But considering that nobodyhas or will ever experience anything without awareness it is illogicalto assume that anything is really outside of awareness. It is actuallyfar more rational to assume that whatever arises or is experienced isinside awareness, and that nothing is outside of awareness. Thisassumption of everything being within awareness would actually be amore scientific, observation-based conclusion than the oppositeassumption which is entirely unfounded on anything we have ever or willever be able to observe. After all, we have never observed anythingapart from awareness have we? Thus contrary to current beliefs, theonus is on scientists to prove that anything is outside of awareness,not the other way around!

Awareness is quite simply theultimate primordial basic nature of reality itself—without awarenessthere could be no “objective reality” at all and no “subjective beings”to experience it. Awareness is completely transcendental, beyond alllimitations and boundaries, outside of all possible systems. Whathubris to think we can simply manufacture, or evolve, awareness with apile of electrified silicon hardware and some software rules.

Nomatter how powerful the computer, no matter what it is made of, and nomatter how sophisticated or emergent the software is, it will stillnever be aware or evolve awareness. No computer or machine intelligencewill ever be aware. Even a quantum computer—if it is equivalent to afinite non-quantum computer at least—will not be capable ofawareness, and even if it is a transinfinite computer I still have mydoubts that it could ever be aware. Awareness is simply not aninformation process.

B. METHOD OF JUDGING THIS BET…

So the question ultimately is, how do we measureawareness or at least determine whether a computer is or is not aware?How can we judge the outcome of this bet?

I propose a method here: we let the bettors mutually agree on a judge.If the judge is a computer, fine. If the judge is a human, fine. Butboth bettors must agree on the judge. If both bettors accept that partyas the judge then the result will be deemed final and reliable. If acomputer is chosen by both parties to judge this, then I will concededefeat—but it would take a lot for any computer to convince me thatit is aware and thus qualified to judge this competition. On the otherhand, my opponent in this debate may accept a human judge—butobviously since they believe that computers can be aware if they accepta human judge they would be contradicting their own assertion—if acomputer is really intelligent and aware why would they choose a humanjudge over a computer judge?

This “recursive” judge-selection judging approach appeals to ourinherent direct human experience of awareness and the fact that wetrust another aware sentient being more than an inaminate machine tojudge whether or not something is aware. This may be the only practicalsolution to this problem: If both parties agree that a computer canjudge and the computer says the other computer is aware, then so be it!If both parties agree that a human can judge and the human says thatthe computer is not aware, so be it! May the best judge win!

Now, as long as we’re on the subject, how do we know that otherhumans, such as our potential human judge(s), are actually aware? Iactually believe that self-awareness is detectable by other beings thatare also aware, but not detectable by computers that are not aware.

C. A REVERSE TURING TEST FOR DETECTING AWARENESS IN A COMPUTER…

Ipropose a reversal of the Turing test for determining whether acomputer is aware (and forgive me in advance if anyone else has alreadyproposed this somewhere, I would be happy to give them credit).

Here is the test: Something is aware if whenever it is presented with acase where a human being and a synthetic machine intelligence areequally intelligent and capable of expression and interaction BUT notequally aware (the human is aware and the machine is not actuallyaware), then it can reliably and accurately figure out that the humanbeing is really aware and the machine is not really aware.

Ibelieve that only systems that are actually aware can correctlydifferentiate between two equally intelligent entities where one issentient and the other just a simulation of sentience, given enoughtime and experience with those systems.

How can such a differentiation be made? Assuming the human andcomputer candidates are equally intelligent and interactive, what isthe signature of awareness or lack of awareness? What difference isthere that can be measured? In my opinion there is a particular, yetindescribable mutual recognition that takes place when I encounteranother sentient being. I recognize their self-awareness with my ownself-awareness. Think of it as the equivalent of a “network handshake”that occurs at a fundamental level between entities that are actuallyaware.

How is this recognition possible? Perhaps it is due tothe fact that awareness, being inherently self-aware, is alsoinherently capable of recognizing awareness when it encounters it.

Onanother front, I actually have my doubts that any AI will ever beequally intelligent and interactive as a human sentient being. Inparticular I think this is not merely a matter of the difficulty ofbuilding such a complex computer, but rather it is a fundamentaldifference between machine cognition and the congition of a sentientbeing.

A human sentient being’s mind transcends computation.Sentient cognition transcends the limits of formal computation, it isnot equivalent to Turing Machine, it is much more powerful than that.We humans are not formal systems, we are not Turing Machines. Humanscan think in a way that no computer will ever be able to match letalone imitate convincingly. We are able to transcend our own logics,our own belief systems, our own programs, we are able to enter andbreak out of loops at will, we are able to know inifinities, to docompletely irrational, spontaneous and creative things. We are muchcloser to infinity than any finite state automaton can ever be. We aresimply not computers, although we can sometimes think like them theycannot really think like us.

In any case, this may be “faith”but for now at least I am quite certain that I am aware and that otherhumans and animals are also aware but that machines, plants and otherinanimate objects are not aware. I am certain that my awareness vastlytranscends any machine intelligence that exists or ever will exist. Iam certain that your awareness is just as transcendent as mine.Although I cannot prove that I am aware or that you are aware to you Iam willing to state such on the basis of my own direct experience and Iknow that if you take a moment to meditate on your own self-awarenessyou will agree.

After all, we cannot prove the existence of spaceor time either—these are just ideas and even physics has notexplained their origins nor can anyone even detect them directly, yetwe both believe they exist, don’t we?

Now if I claimed that asuitably complex computer simulation would someday suddenly containreal physical space and time that was indistinguishable in any way fromthe physical space and time outside the simulation—you would probablydisagree. You would say that the only “real” space-time is actually notin the computer but containing the computer, and any space-time thatappears within the computer simulation is but a mere lower-orderimitation and nothing like the real space-time that contains thecomputer.

No simulation can ever be exactly the same as what itsimulates, even if it is functionally similar or equivalent, forseveral reasons. On a purely information basis, it should be obviousthat if simulation B is within something else called A, then for B tobe exactly the same as A it must contain A and B and so on infinitely.At least if there is a finite amount of space and time to work with wesimply cannot build anything like this, we cannot build a simulationthat contains an exact simulation of itself without getting into aninfinite regression. Beyond this, there is a difference in medium: Inthe case of machine intelligence the medium is physical space, time andenergy—that is what machine intelligence is made of. In the case ofhuman awareness the medium is awareness itself, something at least asfundamental as space-time-energy if not more fundamental. Althoughhuman sentience can perform intelligent cognition, using a brain forexample, it is not a computer and it is not made of space-time-energy.Human sentience goes beyond the limits of space-time-energy andtherefore beyond computers.

If someone builds a Turing Machine that simulates a Turing Machinesimulating a Turing Machine, the simulation will never even start, letalone be useable! As the saying goes, it’s Turtles All The Way Down! Ifyou have a finite space and time, but an infinite initial condition, ittakes forever to simply set up the simulation let alone to compute it.

Thisis the case with self-awareness as well: It is truly self-referential.No finite formal system can complete an infinitely self-referentialprocess in finite time. We sentient beings can do this however.Whenever we realize our own awareness direclty—that is whenever weARE aware (as opposed to just representing this fact as a thought) weare being infinitely self-referential in finite time. That must mean weare either able to do an infinite amount of computing in a finiteamount of time, or we are not computing at all. Perhaps self-awarenessjust happens instantly and inherently rather than iteratively.

On a practical level as well we can see that there is adiffernece between a simulated experience within a simulation and theactual reality it attempts to simulate that exists outside thesimulation. For example, suppose I make a computer simulation ofchocolate and a simulated person who can eat the chocolate. Even thoughthat simulated person tastes the simulated chocolate, they do notreally taste chocolate at all—they have no actual experience of whatchocolate really tastes like to beings in reality (beings outside thesimulation).

Even if there are an infinite number of levels ofsimulation above the virtual reality we are in now, awareness is alwaysultimately beyond them all—it is the ultimate highest-level ofreality, there is nothing beyond it.

Thus even an infinitelyhigh-end computer simulation of awareness will be nothing like actualawareness and will not convince a truly aware being that it is actuallyaware.

Zooming Out in Time

Nova — Mon, 16 Oct 2006 20:22:43 +0000

For an interesting read — download this wonderful presentation on zooming out in time as a way to predict the future. It’s from a talk given at the Long Now Foundation. Nice visual slides illustrate how the world changes over vast timescales.

Visualizing the Tenth Dimension

Nova — Fri, 13 Oct 2006 06:47:42 +0000

One of my readers commented that they were looking for this really cool flash presentation that I blogged about a while back — it helps you visualize higher-dimensions all the way to 10-dimensions. Check it out! After this your brain will need a rest, and possibly a hard reboot — but worth it.

By the way, a reader named Runde has discovered that this visualization is not exactly in accord with the view of string theorists or mainstream physics. You can read the criticisms here. Oh Well, that’s too bad — but I think it’s really cool anyway because it is an exploration of higher dimensions from the perspective of true metaphysics (metaphysics in the philosophical sense, as opposed to new-age metaphysics).

A Proposal to Make the Media (and Society) Better

Nova — Tue, 03 Oct 2006 22:57:26 +0000

I am concerned by what I’m viewing in our national media lately. Viewed from
outside (and also from wihtin the USA), it would appear that our nation
is obsessed with, and plagued by, an increasing spree of horrible crimes and
abuses of human rights. Is this really what it is like to live in
America, or is this simply the media’s extremely biased reporting?

The media of any social organization, whether a small community or an entire nation, serves as the primary means by which the members of that organization understand and assess the state of the whole. As such the media serves a function to a collective that is similar to "self-awareness" in an individual person. Self-awareness is a feedback loop. It not only serves as a mirror by which an organization can view reflections of its state, it also conditions the planning and execution of further actions by its organization. When people look in the mirror of the media, they not only understand the system they are in, but they also may make choices and further actions based on what they see there.

The pivotal role of media as the central influencer of societal psychology and behavior is often underestimated. And it is through this lens that I view the recent headlines, which are primarily focused on school killing sprees and abuse of children as of late — with sadness and concern. By sensationalizing such tragedies, and providing lurid details over and over again, the media is actually contributing to the ongoing proliferation of these same behaviors. While I’m not suggesting that censorship is a better alternative, I do think that leading mass media organizations, in their blind quest for more attention and ad dollars, are behaving irresponsibly by sensationalizing — indeed almost celebrating — murder, torture, sexual abuse, and other horrible acts as if they are the latest attractions in some kind of societal variety show.

These atrocities should be reported, but the style and manner in which they are reported should be the opposite of sensationalism. They should be more like funerals — which is after all what these stories are really about. There should be an element of sadness, respect for those affected, concern for those who might read or view such images and stories, and restraint. In particular details of how various horrible crimes were planned and executed are not necessary and not beneficial in telling the story — they only serve to inspire, trigger and teach would-be copycats to do the same thing. Furthermore by rewarding perpetrators of such crimes with massive publicity (which in many cases is what such egomaniacs are hoping to achieve by committing their crimes) the media is actually playing right into their hands.

As usual I have a radical proposal to fix this: What if the media gave exposure to stories in direct proportion to the number of people actually affected by those stories?

What if stories were evaluated on several dimensions to determine how much exposure to give them — for example, geographic range, number of people directly impacted, political relevance, etc? The size of a story would therefore be determined by its real effect on the population. Contrast this to what the news media does today where in fact the prominence of a story is often inversely proportional to the number of people directly impacted by it. The media currently amplifies the most shocking news, regardless of how many people are directly affected or involved. A shocking crime that directly affected only 2 people is quickly amplified to the main national headline of the day for a population of 300 million. That is simply out of balance with reality.

For example, horrible as they are, the recent school killings only directly affect a relatively tiny percentage of the national population: So why should they be the main focus of the national news headlines for weeks on end? According to this proposal, such stories would be mere blips on the collective radar. They would not be headline news for the entire nation, and/or if they were they would quickly fall out of the headlines. I’m not downplaying the importance of these stories to the affected people and communities — nor am I downplaying the broader significance of the trends they may be indicative of — I’m simply pointing out that these stories, tragic as they may be, get disproportionately too much national media exposure compared to other stories that affect much larger segments of the population such as for example — stories related to the economy, the environment, the war and upcoming elections, etc.

If the news accurately reflected the issues and stories that affect the
population, in proportion to the real impact those issues and stories
have on the population and the number of people directly impacted, then the news would be a lot less sensational,
and a lot more substantial. This would also be better for the
collective psyche. Instead of the media feedback loop focusing
increasingly on sensational atrocities that affect only small numbers
of people, the feedback loop would focus on the larger issues.

Consider this analogy — only a small percentage of the population is
or ever will be criminally insane. Yet the media focuses a relatively
large percentage of its coverage on the crimes comitted by criminally insane people. This has the unfortunate effect of focusing the attention of
the vast majority of non-criminally insane citizens on acts
of criminal insanity. One cannot help but wonder whether by doing this,
the media is actually driving up the amount of criminal insanity in
society? It’s a reasonably hypothesis. And in fact, such effects can be verified
in simple experiments. Whether with oneself or a group of people,
if you obssess about any particular issue over and over again and it will
start to become the main thing everyone thinks about — and this in
turn will start to drive further behavior around that very issue — it’s a self-fulfilling prophecy.

We can see the roles of feedback in driving behavior in other areas as well — For example positive thinking, positive affirmations, positive
feedback, and positive role models lead to more
positive behavior by individuals and groups over time. Similarly, negative thinking, negative
affirmations, negative feedback and negative role models lead to
negative behavior over time. Our media is obsessed with feeding us
negative images of our society, ourselves, and our world — this cannot but have
the effect of generating increasingly negative assessments, experiences
and behaviors that further verify this negative world-view.

It’s a "chicken-and-egg" situation — the media leadership would answer this challenge by saying, "we just report the news. If you want to change the news, change society." But that’s just not accurate. In fact, the media is not an objective reporter, there are all kinds of biases, not the least of which are commercially-motivated. The bottom line, as it’s oft said in newsrooms around the country is, "if it bleeds, it leads." That has nothing to do with objectivity and everything to do with competing for eyeballs, ratings, and ad dollars. There are two ways to fix this chicken-and-egg problem — change the egg (society) or change the chicken (the media.) I am proposing that we change the chicken — which is a lot easier than changing the egg. Getting a few dozen national media organizations to be more objective and less sensationalistic is easier to accomplish than going out and changing the minds and behavior of 300 million people — yet it will have the same long-term effect! And this is the beauty of this approach. Since the media is really functioning as the self-awareness of society, if we recalibrate the media, this will effectively recalibrate society as a whole. The fact is this is already happening — just in a negative direction. It would only require a small effort to make it trend towards the positive.

There are very few positive stories, very few positive role models, and
very little to be happy about in the media today. But this is not an accurate
portrayal of the society or world we live in. In fact, there is so much
that is good in our communities, organizations, societies, and
ourselves. Why is this not portrayed in the media with equal or greater visibility? It’s true that fear,
horror, taboos, crime and conflict grab people by the adrenals and get
their attention — and that leads to short-term attention and ratings gain. But long-term it
leads to societal decay. It’s really no different than a drug addict
chasing after quick fixes at the expense of their long-term health.
Isn’t it time our society started looking at the media feedback loop
more responsibly? And isn’t it time that people start complaining to
the media about their unbalanced and biased coverage of current events?

Unfortunately humans are incredibly short-sighted — we tend to not see
or pay attention to anything outside of the immediate present in space
and time, and when it comes to thinking about societal health we are
particularly myopic. After thousands of years of western civilization
we still do not think about the health of our civilization with
anything close to the models or discipline that we use to think about the
health of the human body. But why don’t we? Why isn’t societal health more of a science today? Why is it still a subject that is relegated to a few non-profits and philosophers on the fringes? Why isn’t it taught in medical schools or the equivalent? Certainly we can and do
measure the state of our society in many dimensions? And we
certainly can see, and prove, the short-term and long-term causal
effects of laws, policies, and even the amount of violence on
prime-time television. So why do we not take this more seriously as a society?

Perhaps it’s the tragedy of the commons at work — everyone feels that
someone else will take care of this, or at least that it is someone
else’s problem. In the end everyone takes care of themseleves and their
own families, but the larger community is left virtually unattended to
– save for those who would take advantage of it for their own (usually
commercial) ends. Eventually the societal psyche becomes so polluted, neglected and abused that it falls apart entirely. This is when you have sudden changes — regime change, economic meltdowns, civil wars, dictatorships, and takeover by extremism and hate-oriented movements — such as the rise of Nazism in Germany in the 1930’s. It is guaranteed that just as is the case with the physical environment, the social environment will also oneday be everyone’s problem, unless we take steps now to guide it in a better direction.

I don’t know how to get the media to report stories in proportion to
their impact, as I have proposed here. But I do know that it would make a
huge difference by reducing the number of sensationally violent and
negative stories in the media every day. This would in turn lead to a happier, more emotionally healthy and less
violent society, with a more positive collective self-image and culture. If in
addition the media actually made an effort to report good news even slightly more than bad news that
would further improve things in dramatic ways.

Imagine the effect on your own psyche by thinking just a few positive, compassionate, life-affirming, happy thoughts every day. Now imagine that times 300 million people. Good news can be just
as entertaining and interesting as bad news, but it has a much more
beneficial long-term effect on the viewer and on society as a whole.

Further Reading:

What Was Before the Big Bang?

Nova — Mon, 04 Sep 2006 22:37:27 +0000

An article in my new favorite magazine, Seed Magazine, by cosmologist Sean Carrol, proposes an interesting new theory about the nature of time and the evolution of baby universes. In this approach, baby universes can suddenly come into being from empty space when random quantum vacuum fluctuations fall into place in just the right way. Admittedly the odds of this happening are incredibly slim, but not impossible, and therefore in an infinite amount of time it definitely will happen over and over again. The only issue I have with the article is that it presents the issue of "time’s arrow" in the wrong light in my opinion. The so-called "arrow of time" is simply the progression from low to high entropy states — that is things start out ordered and become disordered over time. The new proposed theory doesn’t really show any way for that arrow to be reversed as far as I can tell. Entropy doesn’t run backwards, even in the metaverse. It would seem to me that every baby universe would be born from a highly ordered, extremely low entropy, state, and would then become less ordered and would gain in entropy over time. Although this might happen at different moments in meta-time, each of these universes would still develop in the same manner. But perhaps I’m missing something. Maybe the theorists have a way for high-entropy states to suddenly come into being and explode to become full-fledged universes which then lose entropy instead of gaining it? That doesn’t make a lot of sense to me. While the chances of a high-entropy state randomly occuring are incredibly slim, in an infinite amount of time they too would all occur at least some of the time — yet even so, I don’t see any reason to think that a high-entropy baby universe would, or could, run backwards towards a lower entropy state. Feel free to comment and explain it further if I got it wrong.

The Hidden Structure of Quantum Mechanics and The Prime Numbers Turns Out to Be 42 After All

Nova — Mon, 04 Sep 2006 21:59:03 +0000

This is a wonderful article about how a chance encounter led to the discovery of a connection between physics and number theory that may help explain everything from quantum mechanics to the prime numbers….and the most incredible thing is that the answer may actually really be "42" after all. You’ve heard of "Life Imitates Art," well this is "Life Imitates Humor" at it’s best.

Neurons and Universes

Nova — Sun, 20 Aug 2006 06:34:20 +0000

This is a cool pair of images showing a striking similarity between the structure of neurons and that of our universe. I’ve often wondered whether the entire universe isn’t some kind of a mind or a brain in which we are like subatomic particles.

A Tribe That Views Time Differently

Nova — Tue, 13 Jun 2006 20:44:39 +0000

A tribe in South America has been found to have a reverse concept of time from all known cultures:

New analysis of the language and gesture of South
America’s indigenous Aymara people indicates they have a concept of
time opposite to all the world’s studied cultures — so that the past
is ahead of them and the future behind.

Quantum Evolution — A Radical Theory

Nova — Tue, 24 Jan 2006 20:57:58 +0000

The theory of quantum evolution is a radical new take on how mutations
in DNA occur. Basically the theory postulates that DNA molecules are in
fact macroscopic quantum objects that undergo quantum interference. It
is spearheaded by Johnjoe McFadden, a professor in the UK and makes for an interesting
read. Here is a brief overview of the main ideas of the theory. He also has some interesting ideas about a possible interaction between electromagnetic fields and consciousness. It’s way too early to tell whether he is correct in his hypoetheses, but I give him high marks for original thinking! Very interesting stuff.

Collective Intelligence 2.0

Nova — Tue, 24 Jan 2006 19:10:22 +0000

Introduction:

This article proposes the creation of a new open, nonprofit service on the Web that will provide something akin to “collective self-awareness” back to the Web. This service is like a “Google Zeitgeist” on steroids, but with a lot more real-time, interactive, participatory data, technology and features init. The goal is to measure and visualize the state of the collective mind of humanity, and provide this back to humanity in as close to real-time as is possible, from as many data sources as we can handle — as a web service.

By providing this service, we will enable higher levels of collective intelligence to emerge and self-organize on the Web. The key to collective intelligence (or any intelligence in fact) is self-awareness. Self-awareness is, in essence, a feedback loop in which a system measures its own internal state and the state of its environment, then builds a representation of that state, and then reasons about and reacts to that representation in order to generate future behavior. This feedback loop can be provided to any intelligent system — even the Web, even humanity as-a-whole. If we can provide the Web with such a service, then the Web can begin to “see itself” and react to its own state for the first time. And this is the first step to enabling the Web, and humanity as-a-whole, to become more collectively intelligent.

It should be noted that by “self-awareness” I don’t mean consciousness or sentience –I think that the consciousness comes from humans at this point and we are not trying to synthesize it (we don’t need to; it’s already there). Instead, by “self-awareness” I mean a specific type of feedback loop — a specific Web service — that provides a mirror of the state of the whole back to its parts. The parts are the conscious elements of the system – whether humans and/or machines – and can then look at this meta-mirror to understand the whole as wellas their place in it. By simply providing this meta-level mirror, along with ways that the individual parts of the system can report their state to it, and get the state of the whole back from it, we can enable a richer feedback loop between the parts and the whole. And as soon as this loop exists the entire system suddenly can and will become much more collectively intelligent.

What I am proposing is something quite common in artificial intelligence. For example, in the field of robotics, such as when building an autonomous robot. Until a robot is provided with a means by which it can sense itsown internal state and the state of its nearby environment, it cannot behave intelligently or very autonomously. But once this self-representation and feedback loop is provided, it can then react to it’s own state and environment and suddenly can behave far more intelligently. All cybernetic systems rely on this basic design pattern. I’m simply proposing we implement something like this for theentire Web and the mass of humanity that is connected to it. It’s just a larger application of an existing pattern. Currently people get their views of “the whole” from the news media and the government – but these views suffer from bias, narrowness, lack of granularity, lack of real-time data, and the fact that they are one-way, top-down services with no feedback loop capabilities. Our global collective self-awareness — in order to be truly useful and legitimate really must be two-way, inclusive, comprehensive, real-time and democratic. In the global collective awareness, unlike traditional media, the view of the whole is created in a bottom-up, emergent fashion from the sum of the reports from all the parts (instead of just a small pool of reporters or publishers, etc.).

The system Ienvision would visualize the state of the global mind on a number of key dimensions, in real-time, based on what people and software and organizations that comprise its “neurons” and “regions” report to it (or what it can figure out by mining artifacts they create). For example, this system would discover and rank the current most timely and active topics, current events, people, places, organizations, events, products, articles, websites, in the world right now. From these topics it would link to related resources, discussions, opinions, etc. It would also provide a real-time mass opinion polling system, where people could start polls, vote on them, and see the results in real-time. And it would provide real-time statistics about the Web, the economy, the environment, and other key indicators.

The idea is to try to visualize the global mind – to make it concrete and real for people, to enable them to see what it is thinking, what is going on, and where they fit in it – and to enable them to start adapting and guiding their own behavior to it. By giving the parts of the system more visibility into the state of the whole, they can begin to self-organize collectively which in turn makes the whole system function more intelligently

Essentially I am proposing the creation of the largest and most sophisticated mirror ever built – a mirror that can reflect the state of the collective mind of humanity back to itself. This will enable an evolutionary process which eventually will result in humanity becoming more collectively self-aware and intelligent as-a-whole (instead of what it is today– just a set of separate interacting intelligent parts). By providing such a service, we can catalyze the evolution of higher-order meta-intelligence on this planet — the next step in human evolution. Creating this system is a grand cultural project of profound social value to all people on earth, now and in the future.

This proposal calls for creating a nonprofit organization to build and host this service as a major open-source initiative on the Web, like the Wikipedia, but with a very different user-experience and focus. It also calls for implementing the system with a hybrid central and distributed architecture. Although this vision is big, the specific technologies, design patterns, and features that are necessary to implement it are quite specific and already exist. They just have to be integrated, wrapped and rolled out. This will require an extraordinary and multidisciplinary team. If you’re interested in getting involved and think you can contribute resources that this project will need, let me know (see below for details).

Further Thoughts

Today I re-read this beautiful, visionary article by Kevin Kelley, about the birth of the global mind, in which he states:

The planet-sized “Web” computer is already more complex than a human brain and has surpassed the 20-petahertz threshold for potential intelligence as calculated by Ray Kurzweil. In 10 years,it will be ubiquitous. So will superintelligence emerge on the Web, not a supercomputer?

Kevin’s article got me thinking once again about an idea that has been on my mind for over a decade. I have often thought that the Web is growing into the collective nervous system of our species. This will in turn enable the human species to function increasingly as an intelligent superorganism, for example, like a beehive, or an ant colony — but perhaps even more intelligent. But the key to bringing this process about is self-awareness. In short, the planetary supermind cannot become truly intelligent until it evolves a form of collective self-awareness. Self-awareness is the most critical component of human intelligence — the sophistication of human self-awareness is what makes humans different from dumb machines, and from less intelligent species.

The Big Idea that I have been thinking about for over a decade is that if we can build something that functions like a collective self-awareness, then this could catalyze a huge leap in collective intelligence that would essentially “wake up” the global supermind and usher in a massive evolution in its intelligence and behavior. As the planetary supermind becomes more aware of its environment, its own state, and its own actions and plans, it will then naturally evolve higher levels of collective intelligence around this core. This evolutionary leap is of unimaginable importance to the future of our species.

In order for the collective mind to think and act more intelligently it must be able to sense itself and its world, and reason about them, with more precision — it must have a form of self-awareness. The essence of self-awareness is self-representation — the ability to sense, map, reason about, and react to, one’s own internal state and the state of one’s nearby environment. In other words, self-awareness is a feedback loop by which a system measures and reacts to its own self-representations. Just as is the case with the evolution of individual human intelligence, the evolution of more sophisticated collective human intelligence will depend on the emergence of better collective feedback loops and self-representations. By enabling a feedback loop in which information can flow in both directions between the self-representations of individuals and a meta-level self-representation for the set of all individuals, the dynamics of the parts and the whole become more closely coupled. And when this happens, the system can truly start to adapt to itself intelligently, as a single collective intelligence instead of a collection of single intelligences.

In summary, in order to achieve higher levels of collective intelligence and behavior, the global mind will first need something that functions as its collective self-awareness — something that enables the parts to better sense and react to the state of the whole, and the whole to better sense and react to the state of its parts. What is needed essentially is something that functions as a collective analogue to a self — a global collective self.

Think of the global self as a vast mirror, reflecting the state of the global supermind back to itself. Mirrors are interesting things. At first they merely reflect, but soon they begin to guide decisionmaking. By simply providing humanity with a giant virtual mirror of what is going on across the minds of billions of individuals, and millions of groups and organizations, the collective mind will crystallize, see itself for the first time, and then it will begin to react to its own image. And this is the beginning of true collective cognition. When the parts can see themselves as a whole and react in real-time, then they begin to function as a whole instead of just a collection of separate parts. As this shift transpires the state of the whole begins to feedback into the behavior of the parts, and the state of the parts in turns feeds back to the state of the whole. This cycle of bidirectional feedback between the parts and whole is the essence of cognition in all intelligent systems, whether individual brains, artificial intelligences, or entire worlds.

I believe that the time has come for this collective self to emerge on our planet. Like a vast virtual mirror, it will function as the planetary analogue to our own individual self-representations — that capacity of our individual minds which represents us back to ourselves. It will be comprised of maps that combine real-time periodic data updates, and historical data, from perhaps trillions of data sources (one for each person, group, organization and software agent on the grid). The resulting visualizations will be something like a vast fluid flow, or a many particle simulation. It will require a massive computing capability to render it — perhaps a distributed supercomputer comprised of the nodes on the Web themselves, each hosting a part of the process. It will require new thinking about how to visualize trends in such vast amounts of data and dimensions. This is a great unexplored frontier in data visualization and knowledge discovery.

How It Might Work

I envision the planetary self functioning as a sort of portal — a Web service that aggregates and distributes all kinds of current real-time and historical data about the state of the whole, as well as its past states and future projected states. This portal would collect opinions, trends, and statistics about the human global mind, the environment, the economy, society, geopolitical events, and other indicators, and would map them graphically in time, geography, demography, and subject space — enabling everyone to see and explore the state of the global mind from different perspectives, with various overlays, and at arbitrary levels of magnification.

I think this system should provide an open datamodel, and open API for adding and growing data sets, querying, remixing, visualizing, and subscribing to the data.All services that provide data sets, analysis orvisualizations (or other interpretations) of potential value tounder standing the state of the whole would be able to post data into our service for anyone to find and use. Search engines could post inthe top search query terms. Sites that create tag clouds could post intags and tag statistics. Sites that analyze the blogosphere could post in statistics about blogs, bloggers, and blog posts. Organizations that do public opinion polling, market and industry research, trend analysis, social research, or economic research could post instatistics they are generating. Academic researchers could post instatistics generated by projects they are doing to analyze trends on the Web, or within our data-set itself.

As data is pushed to us, orpulled by us, we would grow the largest central data repository aboutthe state of the whole. Others could then write programs to analyze andremix our data, and then post their results back into the system forothers to use as well. We would make use of our data for our ownanalysis, but anyone else could also do research and share theiranalysis through our system. End users and others could also subscribeto particular data, reports, or visualizations from our service, andcould post in their own individual opinions, attention data feeds, orother inputs. We would serve as a central hub for search, analysis,and distribution of collective self-awareness.

The collective self would provide a sense of collective identity: who are we, how do we appear, what are we thinking about, what do we think about what we are thinking about, what are we doing, how well are we doing it, where are we now, where have we been, where are we going next. Perhaps it could be segmented by nation, or by age group, or by other dimensions as well to view various perspectives on these questions within it. It could gather its data by mining for it, as well as through direct push contributions from various data-sources. Individuals could even report on their own opinions, state, and activities to it if they wanted to, and these votes and data points would be reflected back in the whole in real time. Think of it as a giant emergent conversation comprised of trillions of participants, all helping to make sense of the same subject — our global self identity — together. It could even have real-time views that are animated and alive — like a functional brain image scan — so that people could see the virtual neurons and pathways in the global brain firing as they watch.

If this global self-representation existed, I would want to subscribe to it as a data feed on my desktop. I would want to run it in a dashboard in the upper right corner of my monitor — that I could expand at any time to explore further. It would provide me with alerts when events transpired that matched my particular interests, causes, or relationships. It would solicit my opinions and votes on issues of importance and interest to me. It would simultaneously function as my window to the world, and the world’s window to me. It would be my way of participating in the meta-level whole, whenever I wanted to. I could tell it my opinions about key issues, current events, problems, people, organizations, or even legislative proposals. I could tell it about the quality of life from my perspective, where I am living, in my industry and demographic niche. I could tell it about my hopes and fears for the future. I could tell it what I think is cool, or not cool, interesting or not interesting, good or bad, etc. I could tell it what news I was reading and what I think is noteworthy or important. And it would listen and learn, and take my contributions into account democratically along with those of billions of other people just like me all around the world. From this would emerge global visualizations and reports about what we are all thinking and doing, in aggregate, that I could track and respond to. Linked from these flows I could then find relevant news, conversations, organizations, people, products, services, events, and knowledge. And from all of this would emerge something greater than anything I can yet imagine — a thought process too big for any one human mind to contain.

I want to build this. I want to build the planetary Self. I am not suggesting that we build the entire global mind, I am just suggesting that we build the part of the system that functions as its collective self-awareness. The rest of the global mind is already there, as raw potential at least, and doesn’t have to be built. The Web, human minds, software agents, and organizations already exist. Their collective state just needs to be reflected in a single virtual mirror. As soon as this mirror exists they can begin to collectively self-organize and behave more intelligently, simply because they will have, for the first time, a way of measuring their collective state and behavior. Once there is a central collective self-awareness loop, the intelligence of the global mind will emerge and self-organize naturally over time. This collective self-awareness infrastructure is the central enabling technology that has to be there first for the next-leap in intelligence of the global mind to evolve.

Project Structure

I think this should be created as a non-profit open-source project. In fact, that is the only way that it can have legitimacy — it must be independent of any government, cultural or commercial perspective. It must be by and for the people, as purely and cleanly as possible. My guess is that to build this properly we would need to create a distributed grid computing system to collect, compute, visualize and distribute the data — it could be similar to SETI@Home; everyone could help host it. At the center of this grid, or perhaps in a set of supernodes, would be a vast supercomputing array that would manage the grid, do focused computations and data fusion operations. There would also need to be some serious money behind this project as well — perhaps from major foundations and donors. This system would be a global resource of potential incalculable value to the future of human evolution. It would be a project worth funding.

My Past Writing On This Topic

A Physics of Ideas: Measuring the Physical Properties of Memes Towards a Worldwide Database

The Metaweb: A Graph of the Future

From Semantic Web to Global Mind

The Birth of the Metaweb

Are Organizations Organisms?

From Application-Centric to Data-Centric Computing

The Human Menome Project

Other Noteworthy Projects

Principia Cybernetica — the Global Mind Group

The Global Consciousness Project

W3C – The Semantic Web Working Group

Amazon’s Mechanical Turk

CHI — Harnessing Networks of Humans