What does MS use as a User-agent to fetch RSS feeds ?
-
Tuesday, April 28, 2009 8:51 PMWhat does MS use as a User-agent to fetch RSS feeds ?
yahoo seems to use YahooFeedSeeker
google seems to use Feedfetcher-Google
what do you use?
Thanks
All Replies
-
Monday, May 04, 2009 2:33 PMSo just msnbot - i'm presuming. Therefore yahoo and google can still "fetch" and msnbot has been updated in the robots.txt and now blocked.
Seemed to take you 15 hours to honor the robots.txt though.
Thanks -
Monday, May 18, 2009 10:49 PM
Hi Jim,
You are correct. It is just msnbot. I assume you have also been hit hard on your rss feeds. We've just released a patch which made some corrections to our crawl times for the rss feeds. Please email me at lswmc@microsoft.com and I will work with you to correct any remaining issues so that you may unblock us.
Thanks,
Brett
Program Manager, Live Search Webmaster Tools- Marked As Answer by Brett Yount Monday, May 18, 2009 10:49 PM