Comments on: Associative Search and the Semantic Web: The Next Step Beyond Natural Language Search

By: Ben Toth

Ben Toth — Fri, 16 May 2008 09:23:55 +0000

A couple of points in response.
1 The problem with theoretical illustrations of why a new technology is necessary is that they tend to be constructed to illustrate the benefits of the technology, and don’t necessarily reflect real use cases. I once had a new search engine offered to me on the basis that it could distinguish between blind venetians and venetian blinds. We didn’t buy it, but the point is that the use case was completely fanciful. Just how many real world searchers are interested in ‘all products by Sony’?
2 I often have a problem with remembering the names of people in relation to a project . I have a few scraps of asociated information, some of which turn out to be wrong. So I search my e-mail, or my hard disk, and sometimes it takes a few searches. Generally I get there. The point is that I use excisting technology and it doesn’t require the sort of overhead implied above.
The overall point is that the semantic and AI approaches carry a significant overhead, and the illustrations above don’t justify this overhead. I am in favour of better search, and even better metadata (I am a librarian) – but better metadata driven by real use cases.

By: Sherman Monroe

Sherman Monroe — Wed, 14 May 2008 14:30:44 +0000

This is an excellent piece, and important rebuttal to the discussions around Powerset/Hakia/True Knowledge, especially coming from one as yourself who is a proponent of NLP. Lately, I have been attempting to attempting to draw the distinction between what is possible and what is useful in NLP, because there is a difference. As you have observed, NLP is not a solution which provides enhancements in all cases. This is especially true in current web document search. I believe NLP will find its niche as an interface control for accessing and updating data in structured databased, such as RDF repositories, where users are averse to technical query languages. NLP will provide the user a way to use verbs, nouns, adjectives as controls and operators for expressing complex select and update queries. In this way, NLP does for the semantic web, what the GUI did for the command line.
But, again as you correctly identify, as hard as NLP is to fully realize in software, there are still aspects to information retrieval which it is not equipped to address. I believe science fiction has had a lot to do with it, but for whatever reason, people equate the ability to understand natural language as indicating human intelligence in a complete sense, when in fact, a system can be created which correctly parses the Wall Street Journal into formal logical statements, and still be unable to determine whether a particular joke is “funny”. Thus a query for “funny statements made by George Bush” would still be unsolvable (if there is no explicit statement asserting that something he said was funny). The associative aspect of information is one of these very important aspects to point out. The NLP world must be prepared to 1) identify and classify the various modes of information search and retrieval, and in which of those modes can NLP add tremendous value, and 2) focus on providing additional support for user that go a step beyond the answers provided by NLP technology, including research insensitive features such as result set refinement, filtering and faceting.
Thanks so much for the insight.

By: Lars Ludwig

Lars Ludwig — Wed, 14 May 2008 10:10:34 +0000

The differnece between remembering by association and remembering by keyword is similar to the difference between recognition-based search and recall-based search. The SERP offers recognition-based search after initial recall-based search. The suggest box to some degree enables recognition-based search. In my concept of ‘artificial memory’ I found that the personal ‘association’ network is the best starting point for any recall-based searching, because it encloses or points to personal recognizable concepts/things. Semantic search thus would highly benefit from personal knowledge bases.

By: Kingsley Idehen

Kingsley Idehen — Tue, 13 May 2008 20:43:34 +0000

Nova,
Well articulated!
My only quibble is that 15 man years away is too far out. Thus, I am hoping your unit of measure is Internet Time
The Linked Data substrate is already growing organically, the injection of complimentary Lookup oriented Linked Data Spaces is also taking shape as per the UMBEL project.
For example, a corpus of Named Entity Dictionaries + Subject matter Concept schemes + Ontolology level cross mappings gives us a sustrate where “Context” attainment via Subject/Topic aided “Disambiguation” blooms. This also implies that NLP challenges of yore are much reduced.
Historically:
NLP atop SQL just wasn’t/isn’t feasible.
NLP over Unstructured Text (what many are grapping with today over the Web via “Semantic inside” solutions) lacks Contextual richeness, ultimately.
NLP over Structured Linked Data is a whole different ball game. Especially, when you mesh NLP with the Extraction Disambiguation prowess that Linked Data accords. This is basically what’s taking shape now in the Linked Data realm.