Date
17th January, 2005
Presenting
Prof. Alan Dix
Abstract
The protocols of the Internet
and the Web are open and free, the hardware
is distributed and owned by multiple public
and private institutions, the naming and
other central features are administered
by multiple institutions and ultimately
regulated by the UN.
However, the Internet is
not just infrastructure but also services
and perhaps most centrally the ability to
search the Web. In contrast to the highly
decentralised and open nature of the communications
infrastructure, Web search is largely in
the hands of a few large companies, notably
Google, currently the clear market leader,
but challenged by Microsoft. However, given
the costs of web crawling it is not wonder
there are few players out there.
Similarly web directories
are dominated by Yahoo! and the same scaling
issues make it hard to have many players.
However, in contrast to search, the Open
Directory Project (administered by Netscape)
is an 'open' directory in that the editing
is done by volunteers and the database of
classified and annotated pages is available
freely to all for privet, research or commercial
use. In fact Google use this as their web
directory.
Over a period I have been
wondering what it would be like to have
an 'open' web search that used shared, decentralised
resources (along the lines of SETI or gnutlella).
At the same time this may draw in the 90%
of material currently unindexed, the invisible
or hidden web, because it is in web accessible
but not crawlable databases etc.
This has both technical
and commercial problems that would need
to be solve. However these all seem interesting
and challenging to address and have potential
benefits independently of the success of
the wider vision.
Some of the technology
developed in aQtive (the dot.com I was involved
with some years ago) looks promising to
tackle some of the issues and already some
MSc&UG projects have begun to tackle
small parts of the picture.
As well as talking about
some of the technical ideas behind this
I'll also be looking for anyone interested
in working aspects of this or related things.
Downloads
None
|
