An automated program that searches the Internet for new Web documents and places their addresses and content-related information in a database, which can be accessed with a search engine. Generally considered to be a type of bot, or Internet robot.