Sign in
Microsoft.com
 
Resources for IT Professionals
 
 
 
Resources for IT Professionals > Forums Home > Webmaster Forums > Crawling/Indexing Feedback and Discussion > How can I speed up the crawling-speed of msnbot
Ask a questionAsk a question
Search Forums:
 

QuestionHow can I speed up the crawling-speed of msnbot

  • Wednesday, June 10, 2009 8:47 AMjab_doa Users MedalsUsers MedalsUsers MedalsUsers MedalsUsers Medals
     
    Sign In to Vote
    0
    Sign In to Vote
    Hi,

    we have a big site. Currently we have 2,4 Million pages in Google index (+30k pages/day). Google spiders about 200k pages/day. MSNBot only spiders about 500 pages/day. We only have 1,4k pages in bing index. How can we speed up msnbot? Why is it so slow? We do not have crawl-delay set in robots.txt