locked
Site not being crawled or indexed as before Oct 1st RRS feed

  • Question

  • Long standing site dissapeared from your SERPS Oct 1st. and MOST all my pages dissapeared from your INDEX with the exception of my home page. My index page IS showing in the serps.The NEW Algo did not pick up my site pages for the Oct 1st move. MSNbot is crawling 10 to 20 pages a day compared to hundreds before.

     

    My site is all white hat, low KW density, relevant content...etc, ect. There is nothing wrong with my site that would cause this and I'm not alone here. See other forums for many other frustrated webmasters who lost their sites.

     

    We are all scratching our heads over this. The new Algo is definately having problems with crawling and indexing new pages (even old ones) and is also having problems rating the pages for the serps.

     

    We know hard work went into Live Search and we want it to be successful. However we would like to know that these issues are being worked on so our sites will make it back into the serps. Most of us have years of work invested in our sites and to see them dissapear in one day is totally frustrating.

     

    A post from Live Search Team acknowledging the problems and that they are being worked on would go along way in easing our minds.

    Thanks in advance

    Fish Texas

     

     

     

    Wednesday, November 28, 2007 10:17 PM

Answers

  •  PromoManagers.com wrote:

     

    By the way, how would Live even know what is relevant on our site and not?  We have 400 pages, your bot has viewed 2 one of which was the sitemap.  How can your algorithm judge relevance when it won't even look at the pages?

     

    Only part of a relevance algorithm is what is on the page.  Other parts include who links to you and who you link to.

     

     

     PromoManagers.com wrote:

    That's the flaw in the entire process from the MS Team.  The problem isn't a low ranking, the problem is NO ranking.  When I last checked we had no problem with the MS index, we signed up for your tools and we haven't seen your bot in 30 days actually indexing. Ever since we've signed up it comes back and forth and does nothing.

     

    In the last 20 days your bot has pulled livesearchauth.xml 61 times.  61 times?  It's pulled it 27 times in the past 13 days.  That doesn't strike you as a little odd?

     

    hmmm.  I will look into the rate of our auth code pull.  That seems excessive, but it is not connected to the crawler in any way.

     

     PromoManagers.com wrote:

    Here's a few links to similar horror stories including this site which your blog had listed as a "site we read" that got totally delisted because you labeled it as spam.  Are we sure that's not what happened here?  Are we sure your algorithm isn't penalizing corporate names because it doesn't like the name and thinks it's spammy because I'm about sure that's what your problem is with ours.

    http://www.searchenginejournal.com/search-engine-journal-not-indexed-in-microsofts-live-search/4321/

     

    To be fair this issue is almost a year old and was fixed.  It was a problem with spam detection in the old algo.  The current algo uses a different form of spam detection.

     

    thanks for the comments.  

     

    Jeremiah

     

    Wednesday, December 26, 2007 5:02 AM

All replies

  • Hmmm,

     

    I've been chasing this for several days - nice to see that I'm not alone in seeing this; hopefully there is some logical reason for it - until some sort of explanation is presented I am banning msnbot from portions of my site. 

    Thursday, November 29, 2007 1:40 AM
  • I'm pretty sure I've had the same problem. You can find my thread in the Webmaster Tools forum for more details.

    Thursday, November 29, 2007 3:48 PM


  • bro, don't send us digging around and hunting for your thread in the Webmaster Tools forum, post a link to it so it can be seen easily.
    Thursday, November 29, 2007 9:28 PM
  • I was on a terrible connection at the time and would have timed out if I opened up a new tab in IE to find the link. Now that I'm on a better one, here you go: the thread in question.
    Friday, November 30, 2007 12:46 AM
  • We're having similar issues with www.promomanagers.com   It may be helpful if you post your sites during the discussion.  I'm curious about the names of the sites as at least for us I think it has something to do with the results.  The bot is hitting our page and bouncing even though it's extremely compliant.  

     

    I see the bot crawling pages, 10-15 a day, but they're not showing up anywhere.  It's just plain bizzare.

     

    Saturday, December 1, 2007 4:38 AM
  • hi all,

    we're having the same problem on our site www.propertyselect.com

    has anyone got any new information on the problem?
    Saturday, December 1, 2007 7:33 PM
  • More pages are being indexed, slowly...but still nothing in the serps except index page.

    My guess is we won't hear anything on this issue at all.

    Hopefully someone will realize the problem and fix it.

    Hang in there

    Fish Texas

    Saturday, December 1, 2007 7:47 PM
  •  

    I'm having the sam eproblem.

    www.printshopscanada.com has been very well welcomed by yahoo and google , yet we are unable to get more than 2 pages in the Live index. We had a test site prior to this one on a smaller scale that was very well placed on Live search; when we upgraded and changed the domains we added permanent redirects form the old site, but MSNBot seems to have skipped it.

     

    I'm still trying to figure this one out.

    Wednesday, December 5, 2007 10:13 PM
  • Yeap, the same problem here, I've got www.perraco.com being indexed without any problem by Google and Yahoo but Live doesn't seem to like crawling my page.
    Saturday, December 8, 2007 4:15 AM
  • For your site I recommend improving the quality of the inbound links. Many of them do not look like they are related to the content of your website today.

    http://search.live.com/results.aspx?q=%2Blink%3Apromomanagers.com&FORM=ADDLVD&src={referrerTongue Tiedource?}

    regards,
    nate
    Wednesday, December 12, 2007 4:22 AM


  • Honestly shouldn't be no reason for a site to vanish from the search engines.  I don't realy get  why microsoft plays these games.

    One thing is for sure, everyday I find tons  os spamming sites which are not removed from from the Searchs only competitive sites are being removed.

    Now let's think straight.... why competitive sites which sometimes competes with other more powerful company's sites are the ONLY ones being removed?

    Nobody needs to be e genius to figure that out.

    I did report 2  spamming and virus links from keywork's search  which my client's site competed and it totally still there today.   Sad I feel pity for those who is clicking in there.

    Only good and well worked sites are being removed from the msn

    http://forums.microsoft.com/webmaster/ShowPost.aspx?PostID=2577747&SiteID=79


    Merry Christmas to all of you and Thank God MSN is not the biggest search engine.

    that's is already a relieved.

    Saturday, December 22, 2007 2:15 AM
  • Same thing happened to my wifes site, I got a generic response about quality site construction. I'll tell you this if you search for Palm Coast Photography in MSN one of the top 5 sites in the results is definately running a domain farm and using hidden text in their page, yet msn is ok with that. I've dedicated ALOT of time making sure there are no errors or shady tatctics involved with my wifes web page and it seems that abiding by the rules has gotten me nowhere with MSN.

     

    http://forums.microsoft.com/webmaster/ShowPost.aspx?PostID=2545413&SiteID=79

    Saturday, December 22, 2007 4:36 AM
  •  

    Nathan, what would Microsoft consider a relevant link?  Should our competitors put up links for us on their site and vice versa?  Should our vendors put up links for all 20,000 of their customers?  If we did that you'd be telling us that's the problem.   The reason MS sees irrelevant links is because live bot is incompetent in finding the relevant links which tells me, and should tell you and the team that the problem is YOUR bot. 

     

    There are relevant links out there from authority sites that MS hasn't indexed in weeks.  It's astounding.

     

    Consider that every XP/Vista machine ships with Live/MSN as the default search in IE yet you are still bleeding market share to Google.  That tells me overwhelmingly that the rest of the world still finds the SERPs as irrelevant as the site operators are finding. 

     

    Your new algorithm cannot tell the difference between promomanagers and promo managers in a search.  All the backlinks in the world won't change the inability of live search to comprehend plain language.

     

    I'll ask again, what is a relevant link for Microsoft?  Our problem has zero to do with inlinks, what you and Jeremy are not understanding is for some reason your bot has STOPPED indexing sites.  It visits sites every single day, it visits ours maybe 5-10 times a WEEK but it won't index anything.  Something is causing it to repeatedly come back and repeatedly leave and I am betting $50 it's the webmaster tools.   Something is failing after it sees the site is authorized, it comes back and checks that file sometimes 4 times a day.

     

     

     

    Saturday, December 22, 2007 5:28 PM
  •  

    By the way, how would Live even know what is relevant on our site and not?  We have 400 pages, your bot has viewed 2 one of which was the sitemap.  How can your algorithm judge relevance when it won't even look at the pages?

     

    That's the flaw in the entire process from the MS Team.  The problem isn't a low ranking, the problem is NO ranking.  When I last checked we had no problem with the MS index, we signed up for your tools and we haven't seen your bot in 30 days actually indexing. Ever since we've signed up it comes back and forth and does nothing.

     

    In the last 20 days your bot has pulled livesearchauth.xml 61 times.  61 times?  It's pulled it 27 times in the past 13 days.  That doesn't strike you as a little odd?

     

    Here's a few links to similar horror stories including this site which your blog had listed as a "site we read" that got totally delisted because you labeled it as spam.  Are we sure that's not what happened here?  Are we sure your algorithm isn't penalizing corporate names because it doesn't like the name and thinks it's spammy because I'm about sure that's what your problem is with ours.

    http://www.searchenginejournal.com/search-engine-journal-not-indexed-in-microsofts-live-search/4321/

    Saturday, December 22, 2007 5:45 PM
  •  PromoManagers.com wrote:

     

    By the way, how would Live even know what is relevant on our site and not?  We have 400 pages, your bot has viewed 2 one of which was the sitemap.  How can your algorithm judge relevance when it won't even look at the pages?

     

    Only part of a relevance algorithm is what is on the page.  Other parts include who links to you and who you link to.

     

     

     PromoManagers.com wrote:

    That's the flaw in the entire process from the MS Team.  The problem isn't a low ranking, the problem is NO ranking.  When I last checked we had no problem with the MS index, we signed up for your tools and we haven't seen your bot in 30 days actually indexing. Ever since we've signed up it comes back and forth and does nothing.

     

    In the last 20 days your bot has pulled livesearchauth.xml 61 times.  61 times?  It's pulled it 27 times in the past 13 days.  That doesn't strike you as a little odd?

     

    hmmm.  I will look into the rate of our auth code pull.  That seems excessive, but it is not connected to the crawler in any way.

     

     PromoManagers.com wrote:

    Here's a few links to similar horror stories including this site which your blog had listed as a "site we read" that got totally delisted because you labeled it as spam.  Are we sure that's not what happened here?  Are we sure your algorithm isn't penalizing corporate names because it doesn't like the name and thinks it's spammy because I'm about sure that's what your problem is with ours.

    http://www.searchenginejournal.com/search-engine-journal-not-indexed-in-microsofts-live-search/4321/

     

    To be fair this issue is almost a year old and was fixed.  It was a problem with spam detection in the old algo.  The current algo uses a different form of spam detection.

     

    thanks for the comments.  

     

    Jeremiah

     

    Wednesday, December 26, 2007 5:02 AM