locked
Live Crawling Our Site As https RRS feed

  • Question

  •  

    Please help - Live is crawling my clients site - but is crawling https!

    All of our listed pages are showing up in Live Search as https

     

    How do we get this to stop and just spider our http?

     

    Please help!

     

    Shea

     

    Webmaster

    http://www.BasketsforHer.com

    Friday, March 14, 2008 3:16 PM

Answers

  • Use a sitemap.xml file posted in your root directory. Inside the sitemao.xml file you can tell the robots which directories they are NOT to crawl. There are free programs on the net that will create this file for you. Most webmaster tool sets will validate your file.... google has the best tools I've seen so far.

     

    Friday, March 14, 2008 6:06 PM