Web Crawling (Date Crawling using c#)
A web crawler (also known as a web spider or web robot) is a program or automated script which browses the World Wide Web in a methodical, automated manner.. Web crawling is a very advance field for researchers in the field of data mining. Almost all modern languages provide easy to use methods for implementing web crawling. Search Engine actually is an information retrieval system that helps users to find information stored on computer system or systems. Fig: 01 Web crawling picture courtesy:www.sayonetech.com The search results commonly known as “hits” are presented in the form of a list to the users. The current search engines like Google, Yahoo and MSN hits millions of records against a single query. Among these millions of records it’s very difficult and time consuming for the users to find the relevant information. These search engines search information based on key words mentioned in the query. Date sensitive search engines have the capability to give pr...