I get errors in my tomcat log from msnbot/2.0b requests. They concern a set of urls that end with the pattern \t\t\t (i guess these represent three tabs). I have tried to locate these urls in my site, but I can't find them, which makes me believe they may either be entered by hand, or scrambled by the bot, especially as the same links are requested and over again. The urls are (as far as I can determine):
I thought I had replied to this already, but I'm not sure what happened. I submitted a URL removal request a day or so after your post to block that specific URL (with the t\t\t etc). If it is still showing, let me know and I will dig deeper into the issue.
Thanks,
Brett
Program Manager, Live Search Webmaster Tools
Marked as answer byBrett YountWednesday, February 11, 2009 10:57 PM