Writing a Web Crawler in the Java Programming Language
Hits: 103
Description: Everyone uses web crawlers—indirectly, at least! Every time you search the Internet using a service such as Alta Vista, Excite, or Lycos, you're making use of an index that's based on the output of a web crawler. Web crawlers—also known as spiders, robots, or wanderers—are software programs that automatically traverse the Web. Search engines use crawlers to find what's on the Web; then they construct an index of the pages that were found.
