Tuesday, April 28, 2009 8:51 PMWhat does MS use as a User-agent to fetch RSS feeds ?
yahoo seems to use YahooFeedSeeker
google seems to use Feedfetcher-Google
what do you use?
Monday, May 04, 2009 2:33 PMSo just msnbot - i'm presuming. Therefore yahoo and google can still "fetch" and msnbot has been updated in the robots.txt and now blocked.
Seemed to take you 15 hours to honor the robots.txt though.
Monday, May 18, 2009 10:49 PMHi Jim,
You are correct. It is just msnbot. I assume you have also been hit hard on your rss feeds. We've just released a patch which made some corrections to our crawl times for the rss feeds. Please email me at email@example.com and I will work with you to correct any remaining issues so that you may unblock us.
Program Manager, Live Search Webmaster Tools
- Marked As Answer by Brett Yount Monday, May 18, 2009 10:49 PM