Introduction - If you have any usage issues, please Google them yourself
Web crawler (also known as web spider, robot, in the middle of the FOAF community, more often referred to as Web Chaser), is one kind of in accordance with certain rules, automatic program or script to crawl the World Wide Web information. Do not use the name there are ants, automatic indexing, the simulation program or worm.