Which of the following is NOT a function of the robots.txt file?

  • Show search engine bots all the pages on a site
  • Prevent search engines from crawling private pages
  • Keep search engine bots from overwhelming server resources
  • Specify the location of your sitemap

Leave a Reply

Your email address will not be published. Required fields are marked *