locked
MSNBot and odd search terms RRS feed

  • Question

  • I'm seeing search keyphrases like "search", "register", and "password" as search engine referral keywords in my logs. It's from the same set of IP addresses (don't have access to those address at the moment, however) that others have said are from MSN/Live. I have been seeing this for over a year, and others have seen it too.

    Vanessa Fox wrote about this on Search Engine Land in December 2007, and her statement of one of the problems MSNBot was causing site owners was "
    Added unrelated search terms to HTTP logs: The bot often crawled a page as a user clicking through a search result. The search queries the bot used were often not related to the site, which caused incorrect search query reports and perplexed many a webmaster.". This is what I am seeing. In her post, she said that there was a blog post (I believe from Microsoft) that The bot...won't use non-related search strings." It's still happening to me.

    http://searchengineland.com/microsoft-reports-that-issues-with-their-live-search-cloaking-detection-system-have-been-fixed-12843

    I'm tempted to block MSNBot entirely, considering how it is skewing the logs, using lots of bandwidth (even though I've put a crawl-delay of 5 in robots.txt), and we have very few visitors from MSN/Live.

    Any suggestions?
    Tuesday, January 13, 2009 7:12 PM

Answers

  • Hi Morgret,

    are you still experiencing these issues? I know the team involved is working to correct this.


    Thanks,

    Brett
    Program Manager, Live Search Webmaster Tools
    • Marked as answer by Brett Yount Tuesday, March 31, 2009 8:12 PM
    Tuesday, March 31, 2009 8:12 PM

All replies

  • Hi Morgret,

    I'm still waiting on a response from the team handling these issues. I will update the thread as soon as I get a response.\

    Thank you for your patience.


    Brett
    Program Manager, Live Search Webmaster Tools
    Wednesday, February 11, 2009 8:29 PM
  • Hi Morgret,

    are you still experiencing these issues? I know the team involved is working to correct this.


    Thanks,

    Brett
    Program Manager, Live Search Webmaster Tools
    • Marked as answer by Brett Yount Tuesday, March 31, 2009 8:12 PM
    Tuesday, March 31, 2009 8:12 PM
  • Hey Brett,

    Found this thread via searching your forums for an answer to these problems. I'm having them also.

    Any idea what's causing them?

    Mike.
    • Edited by mike_dr Wednesday, April 8, 2009 11:08 AM Typo.
    Wednesday, April 8, 2009 11:07 AM