locked
Not Crawling? RRS feed

  • Question

  • Our weblogs indicate that the msnbot is not crawling our site and it appears that numerous others are experiencing the same problem.  I sent emails to support, called, posted to the forum and received 1 response that said your site is not ranked well in the forum all other correspondence was ignored.

    No crawling = no indexing possible = will never rank.

    If the msnbot will not crawl your site it does not matter what you do because the site will never index unless microsoft has developed the osmosis or telepathic bot.

    Conclusions:  It is a wasted effort trying to optimize for live search.  Wasted time, wasted pages and lost revenue.

    Will not crawl consequences:

    As a result, I am rapidly optimizing pages away from the msnbot to other search engines and live can pick up the crumbs.

    I have come to the conclusion that msn is not giving me the most relavant information and have changed my search engine after some 10 years.

    Finally,  I manage a stock portfolio and am seriously considering selling the msft securities because it would seem to me that a failure to crawl will result in reduced revenues for microsoft and I see no way that they can ever catch google.
     
    All microsoft had to do was acknowledge the problem now the pages are leaving and no longer compatible with their algorithms and gone for good are more pages everyday.  We are also developing another site and have come to the conclusion that there is no reason to optimize pages for the msnbot for that site either.

    We are willing to play the game but if the msnbot will not crawl the efforts are usless.
    Monday, February 2, 2009 5:11 PM

Answers

  • There is a lot more involved in site ranking than on-page elements. Although content is king, so too are relevant backlinks. Increase your backlinks and you will find that your rank will increase.


    Brett



    Program Manager, Live Search Webmaster Tools
    • Marked as answer by Brett Yount Wednesday, February 11, 2009 10:54 PM
    Tuesday, February 3, 2009 8:01 PM

All replies

  • Tax Guru,

    Unfortunately, your site is not ranked well. I show that you were crawled 1/18/2009. I also show that out of the 13 backlinks to your site, several are from directories which do not help your rank. Again, I suggest reading our FAQs located at the top of the indexing and ranking forums as well as some of the blog posts located at http://blogs.msdn.com/webmaster which will provide a better understanding of how our crawler works and what you can do to help increase ranking, crawl delays and indexing.

    Brett 
    Program Manager, Live Search Webmaster Tools
    Monday, February 2, 2009 6:38 PM
  •  The crawl you speak of I believe came from a resubmit and the crawler stopped at that page, indexed that page and did rank well for some keyword phrases so why was the page deleted in the first place?  In additon, our weblog shows 0 page accesses by msnbot.  This is possible though because our weblogs show only those that access the most.  We are watching to see if our weblogs indicate microsoft has crawled more than one page, if at all.  We are already, aware that the algorithms have changed and have no problem with that but we are convinced that the weblogs are not unable to detect the msnbot.  As to the rankings, you can not index or rank if the bot does not crawl and many pages were changed so there is no reason for the bot not to crawl and without crawling there is no way for the bot to determine whether the algorithms are correct or not.

    Keeps messing aroung there will not be anything compatible for the bot to index.  We are watching the weblogs daily and time will tell.  It is absolutely amazing to me that the algorithms changed so drastically that everything just disappeared when we had volumnes of ranked pages one day and everything gone the next.  I suppose there is an explanation but why did not any other sites disappear for the same keywords and keyword phrazes this is also suspicious..

    Another suspicious indication is that so few pages even show up when we search our address even if the do not rank.  One day indexed 30 pages next day gone were not even compatible enough to index, overnight.  Like, I said before and poof it was gone. 

    If we have an optimaztion issue that we can work with but if the bot refuses to crawl the site we can not do anything about that.

    Monday, February 2, 2009 9:05 PM
  • There is a lot more involved in site ranking than on-page elements. Although content is king, so too are relevant backlinks. Increase your backlinks and you will find that your rank will increase.


    Brett



    Program Manager, Live Search Webmaster Tools
    • Marked as answer by Brett Yount Wednesday, February 11, 2009 10:54 PM
    Tuesday, February 3, 2009 8:01 PM