17 กรกฎาคม 2551 2:19I created a Sitemap over a month ago and my server logs show that the MSNbot is visiting my website. However, only my home page is listed in the index and under my Webmaster Tools, it says my site was last crawled on 1/14/2008. Why does it say my site was crawled back in January, but my server logs shows 35 page views in the last week from the MSNbot?
My site is well-designed and well ranked in Yahoo and Google, so I'm not using black-hat techniques at all, so I'm not sure why none of my other pages are not being found? My Webmaster Tools say my page is not blocked.
My site is: http://www.cheatsheetwarroom.com
21 กรกฎาคม 2551 16:02
You might try adding a robots.txt file with a path to your sitemap.
26 กรกฎาคม 2551 20:49I went ahead and added one, but I'm thinking that my problem is that my site hasn't been crawled since January. Any idea why that would be?
30 กรกฎาคม 2551 17:37
it seems like the msn crawler hits something it dont like.. it seems to try coming back to see if you have solved the problem (according to youre log). since you dont have solved it, it dont update the last crawled date. Look into youre coding, and see if its healthy. (script, placement of script, tags, "air" inside coding++)