Sign in
Microsoft.com
 
Microsoft
 
 
 
Microsoft >
> 
> 
How can I speed up the crawling-speed of msnbot

Locked How can I speed up the crawling-speed of msnbot

  • 2009. június 10. 8:47
     
     
    Bejelentkezés a szavazáshoz
    0
    Hi,

    we have a big site. Currently we have 2,4 Million pages in Google index (+30k pages/day). Google spiders about 200k pages/day. MSNBot only spiders about 500 pages/day. We only have 1,4k pages in bing index. How can we speed up msnbot? Why is it so slow? We do not have crawl-delay set in robots.txt