How to be a Internet-wide search engine
FDSE is a single-site or cluster-of-sites search engine. It can hold a few thousands URLs and search them quickly.
FDSE does not compete with major search engines like Yahoo!, Google, or Altavista.
If you wish to create a search engine that can search the entire Internet, try these products and services:
- Google Custom WebSearch
- Ultraseek (formerly Inktomi Enterprise Search)
The full Internet search solutions tend to be expensive. You can also use a "meta search" script; see Running as a "meta search" to display results of other search engines for information on that.
"How to be a Internet-wide search engine"
http://www.xav.com/scripts/search/help/1008.html