Live Crawling Our Site As https RRS feed

  • Question


    Please help - Live is crawling my clients site - but is crawling https!

    All of our listed pages are showing up in Live Search as https


    How do we get this to stop and just spider our http?


    Please help!






    Friday, March 14, 2008 3:16 PM


  • Use a sitemap.xml file posted in your root directory. Inside the sitemao.xml file you can tell the robots which directories they are NOT to crawl. There are free programs on the net that will create this file for you. Most webmaster tool sets will validate your file.... google has the best tools I've seen so far.


    Friday, March 14, 2008 6:06 PM