locked
Why won't MSNBot crawl and index more of my pages RRS feed

  • Question

  • Hi everyone,
      my website has been listed on MSN for sometime now, and its currently only got 543 indexed pages. The site is pretty successful with over 200,000 pages indexed on Google and many thousands in Yahoo, so I'm baffled as to why MSNBot won't index more pages?  I submitted a sitemap many many months ago, but it doesn't seem to make any difference.  I have plenty of sites linking to ConcertHotels.com, so why doesn't MSNBot want to list any of my pages? Please help!

    Many thanks
    Mike
    Wednesday, November 26, 2008 12:27 PM

Answers

  • @ Alray,

     

    The Page and Domain Scores are not the same as PageRank.

     

    Per help text located at: http://help.live.com/help.aspx?mkt=en-us&project=wl_webmasters

  • Domain score

    Provides a measurement of how authoritative Live Search views your domain to be, with five green boxes being the highest rating and five empty boxes being the lowest. This is based on many of the same factors Live Search uses to determine static rank, but isn't directly comparable.

  • Page score

    Provides a measurement of how authoritative Live Search views your webpage to be, with five green boxes being the highest rating and five empty boxes being the lowest. This is based on many of the same factors Live Search uses to determine static rank, but isn't directly comparable.

     

    Depth of crawl is based on a lot of factors including your site's static rank and the architecture of your site.  However, here are some steps you can take to help us crawl your site:

    ·         Verify that you are not blocked in Webmaster Tools and

    o   There are no site crawling errors

    ·         Make sure you are not blocking MSNbot in your robots.txt file

    ·         Make sure you are not blocking Msnbot’s IP range

    ·         Ensure there are no site navigation issues that are preventing MSNbot from crawling your site (partially found in Webmaster Tools):

    o   Invalid coding (HTML/XHTML, etc.) validate your code at www.w3c.org

    o   Site navigation links that are only accessible by JavaScript

    o    Password protected

    ·         Submit a sitemap.xml file in Webmaster tools and provide the path to the sitemap(s) in a robots.txt file

    A resource to review:

    http://www.seomoz.org/blog/back-to-basics-site-architecture-issues-to-avoid#jtc8342

     

Monday, December 1, 2008 6:33 PM

All replies

  • MSN only indexes a few pages from your site for the simple reason, MSN is broke, I suspect it could be fixed by most of the webmasters that post on here. The problem is MSN are to stubborn and stupid to admit it is broke so will not fix it.
    I suggest you spend your time working on Google and Yahoo, as MSN is now no longer a major player, Your time would be better spent working on a search Engine like Dogpile.
    Monday, December 1, 2008 3:11 AM
  • Well as far as SE is concerned live search is still a major player in the industry. Each SE has their own algo, I have this site been visiting by bot but is not being indexed yet, but I least I know they will be indexed.
    Monday, December 1, 2008 6:14 AM
  • MSN a major player? are you unaware they are not even top ten. You don't know you will be indexed, I can assure you if you are indexed MSN will index just a fraction of your pages.

    MSN bot visits our site everyday and have done for sometime it indexes about 10% of our site, even though they rank us as a PR5.

    Monday, December 1, 2008 6:42 AM
  • First MSN does not rank site by PR, hope you know that. Look search industry ranking, the are considered in the big 3.

    Monday, December 1, 2008 2:51 PM
  • In Webmaster tools they give page rank. There are many search engine rankings in some they still are 3 in many they are way down and in some not even in the top ten. 

     

    Tell me what percentage of hits does your site get from Live and MSN, I will be shocked if its above 2%
    Monday, December 1, 2008 3:20 PM
  • @ Alray,

     

    The Page and Domain Scores are not the same as PageRank.

     

    Per help text located at: http://help.live.com/help.aspx?mkt=en-us&project=wl_webmasters

  • Domain score

    Provides a measurement of how authoritative Live Search views your domain to be, with five green boxes being the highest rating and five empty boxes being the lowest. This is based on many of the same factors Live Search uses to determine static rank, but isn't directly comparable.

  • Page score

    Provides a measurement of how authoritative Live Search views your webpage to be, with five green boxes being the highest rating and five empty boxes being the lowest. This is based on many of the same factors Live Search uses to determine static rank, but isn't directly comparable.

     

    Depth of crawl is based on a lot of factors including your site's static rank and the architecture of your site.  However, here are some steps you can take to help us crawl your site:

    ·         Verify that you are not blocked in Webmaster Tools and

    o   There are no site crawling errors

    ·         Make sure you are not blocking MSNbot in your robots.txt file

    ·         Make sure you are not blocking Msnbot’s IP range

    ·         Ensure there are no site navigation issues that are preventing MSNbot from crawling your site (partially found in Webmaster Tools):

    o   Invalid coding (HTML/XHTML, etc.) validate your code at www.w3c.org

    o   Site navigation links that are only accessible by JavaScript

    o    Password protected

    ·         Submit a sitemap.xml file in Webmaster tools and provide the path to the sitemap(s) in a robots.txt file

    A resource to review:

    http://www.seomoz.org/blog/back-to-basics-site-architecture-issues-to-avoid#jtc8342

     

Monday, December 1, 2008 6:33 PM
  • @Brett

     

    So it shows something else on MSN that does not work our Domain score is 5. Many Page score 5.

    but you still only index a fraction of the pages.

     

    PLEASE PLEASE PLEASE.

     

    Don't try feeding me rubbish tell the truth, MSN IS BROKEN.

     

    Monday, December 1, 2008 9:39 PM