Distributed search engine

A distributed search engine is a search engine where there is no central server. Unlike traditional centralized search engines, work such as crawling, data mining, indexing, and query processing is distributed among several peers in a decentralized manner where there is no single point of control.

This section may require cleanup to meet Wikipedia's quality standards. The specific problem is: Sub-section order is not chronological, but potentially promotional. Please help improve this section if you can. (February 2025) (Learn how and when to remove this message)

Started in 2017, Presearch is an ERC20 powered (PRE) search engine powered by a distributed network of community operated nodes which aggregate results from a variety of sources. This powers the searches at presearch.com. This is planned to be a precursor where each node collaborates on a global decentralised index. ^[1] Presearch averages 5 million searches per day and has 2.2 million registered users. On Sept 1, 2021, Presearch was added as a default option to the search engine list on Android for the EU.^[2] On May 27, 2022, Presearch officially transitioned from its Testnet to a Mainnet. This means all search traffic through the service now runs over Presearch's decentralized network of volunteer-run nodes.^[3]

On December 15, 2003, Michael Christen announced development of a P2P-based search engine, eventually named YaCy, on the heise online forums.^[4][5]

Seeks was an open source websearch proxy and collaborative distributed tool for websearch. It ceased to have a usable release in 2016.

In April 2000 several programmers (including Gene Kan, Steve Waterhouse) built a prototype P2P web search engine based on Gnutella called InfraSearch. The technology was later acquired by Sun Microsystems and incorporated into the JXTA project.^[6] It was meant to run inside the participating websites' databases creating a P2P network that could be accessed through the InfraSearch website.^[7][8][9]

On May 31, 2000 Steelbridge Inc. announced development of OpenCOLA a collaborative distributive open source search engine.^[10] It runs on the user's computer and crawls the web pages and links the user puts in their opencola folder and shares resulting index over its P2P network.^[11]

In February 2001 Wolf Garbe published an idea of a peer-to-peer search engine,^[12] started the Faroo prototype in 2004,^[13] and released it in 2005.^[14][15]

This section does not cite any sources. Please help improve this section by adding citations to reliable sources. Unsourced material may be challenged and removed. Find sources: "Distributed search engine" – news · newspapers · books · scholar · JSTOR (February 2025) (Learn how and when to remove this message)

The goals of building a distributed search engine include:

1. to create an independent search engine powered by the community;

2. to make the search operation open and transparent by relying on open-source software;

3. to distribute the advertising revenue to node maintainers, which may help create more robust web infrastructure;

4. to allow researchers to contribute to the development of open-source and publicly-maintainable ranking algorithms and to oversee the training of the algorithm parameters.

This section does not cite any sources. Please help improve this section by adding citations to reliable sources. Unsourced material may be challenged and removed. Find sources: "Distributed search engine" – news · newspapers · books · scholar · JSTOR (February 2025) (Learn how and when to remove this message)

1. The amount of data to be processed is enormous. The size of the visible web is estimated at 5PB spread around 10 billion pages.

2. The latency of the distributed operation must be competitive with the latency of the commercial search engines.

3. A mechanism that prevents malicious users from corrupting the distributed data structures or the rank needs to be developed.

• List of search engines § P2P search engines
• Distributed processing