Comparison of Three Vertical Search Spiders
(2003) Comparison of Three Vertical Search Spiders. Computer 36(5):pp. 56-62.
Full text available as: |
Abstract
Spiders are the software agents that search engines use to collect content for their databases. We investigated algorithms to improve the performance of vertical search engine spiders. The investigation addressed three approaches: a breadth-first graph-traversal algorithm with no heuristics to refine the search process, a best-first traversal algorithm that used a hyperlink-analysis heuristic, and a spreading-activation algorithm based on modeling the Web as a neural network.
| EPrint Type: | Journal Article (Paginated) |
|---|---|
| Keywords: | National Science Digital Library, NSDL, Artificial Intelligence Lab, AI Lab, Web spider |
| Subjects: | Web Mining Internet Data Mining |
| ID Code: | 413 |
| Deposited On: | 16 August 2004 |
| Alternative Locations: | http://ai.bpa.arizona.edu/go/papers.html |
| Eprint Statistics: | View statistics for this eprint |
| Tell A Colleague: | Tell a colleague about it. |