locked
One more try re how live searches RRS feed

  • Question

  • Jeremiah I asked this on the other thread but didn't get an answer.

     

    In a previous example with our site this site shows up #1 when searching for promomanagers.com.

     

    http://www.hilton.com/en/es/hotels/hotelpromo.jhtml?ctyhocn=WASCCES&promo=Managers_Reception&cid=OH,ES,wascc,ManagersB

     

    There is absolutely no mention anywhere in the content of Promo or Managers, and certainly not the two together.  The only place the words are proximal is in the url yet it ranks number one ahead of all other results.  Does this mean the key to happines with Live is to stuff your url with keywords?  What good will any of the recommendations you've given all of us regarding content and indexability do if a high ranking site in a totally unrelated field happens to put up a url that you index related to a company name one of us owns? 

     

    Is this issue going to be addressed or is it even being understood? 

     

    Point blank, why are a bunch of our sites only located by typing in site:www.ournames.com?

     

    Why does MSN visit and index 394 pages in 3 days and yet never update the index? It's done this now weekly, visiting more pages each time and yet none ever get into the index? 

     

    Thanks

    Saturday, January 5, 2008 3:44 PM

Answers

  •  PromoManagers.com wrote:

    Does anyone at Microsoft take responsibility for what's happening and have an actual answer to why your bot crawls 700 pages but indexes none?  Can someone take actual responsibility for this instead of Global sending us here where we get no answers?

     

    As stated in the past there is a limit to the size of our index and while we would love to index every page on the web and would love to index every site.  We currently cannot guarentee that each site will be indexed and we don't guarentee coverage to any site. 

     

    We are growing our index all the time. We quadrupled our index size in the last year and hopefully we will continue to grow the same way next year.     So why does it crawl 700 pages and index none.  Because we have to select what we think are the best pages.  Not everyone will agree with the design of the selection and ranking algorithm and we are always working on making selection better.

     

    Jeremiah

    Monday, January 14, 2008 7:37 PM
  •  Fergi wrote:


    Question - If a site's coding has been validated by the W3C, which forgive me if I'm wrong here is something of an authority on the subject of correct coding, is there some very strange algorithms going on in their search engine?.
    Question - So if a sites not blocked (which mine isn't because MSN Support have said it isn't)) and the coding is OK (which I regard mine as OK see above) the robots.txt file (which MSN Support says is OK and which apparently Livesearch seems to rely on to index pages). My sitemap.xml file has been crawled (because my site logs have registered that it has) So who has the problem?


     

    Fergi,

     

    Let me answer the qustions first if possible.   Your code will very rarely have an impact on rank unless you are using Flash, Silverlight, or AJAX like scripting without providing an alternate method of navigation.  Design of a site might have an impact, but I looked at your site and it appears fine.

     

    Your second question is the "question of the ages"  In my mind you are asking "I am doing everything right, but I am not ranking?"

     

    So I took a look at some of the site linking to you...  You can see these in our webmaster tools or in any of the other search engine tools.

     

    Here are the top ten at the moment I looked at your site:

    www.turnpike.net/directory.html?showcat+home+280
    www.turnpike.net/directory.phtml?showcat+home+240 
    metro.turnpike.net/directory.phtml?showcat+home+22
    metro.turnpike.net/directory.phtml?showcat+home+24
    www.etkal.com/directory/Homemaking/Homemaking-1.ht
    www.zabalook.com/Home/Consumers.php?a=19 - SP
    www.excellentweb.info/directory/Consumer-Informati
    www.esgsite.com/index.php?c=391&p=7 
    www.cocovida.co.uk/links/rss.php?c=9
    www.fatinfo.com/home/Tumble-Dryer-repairs-in-Kent

     

    Following these links they look mostly like directorys which don't really have any authoritative rank in the case of a couple actually appear to be link farms.  These are not the kind of links that will help you build rank infact they may hurt you.

     

    Quality links can be directories, but the content and context of the directory needs to be really good. (see industry association below). If you still want to build links with directories, I recommend being very choosy. You could looking into the following link building methods.

    • Join a reputable industry association – often times they will list their members and provide links to their websites. Examples could be a local rotary club, or a professional association like the American Medical Association.
    • Get involved with your community – participating with your community through blogs, forums and other online resources may give you legitimate reasons to provide links to your site.
    • Press Releases – if your company has a significant event, consider doing a press release through a site like http://prweb.com.
    • Suppliers and partners – ask your business partners if they would add a section to their website describing your partnership, with a link to your website. Or, if your suppliers have a website, perhaps they have a page where they recommend local distributers of their products.

    The reality is that you are in the index but your site is not ranking well.   Your sitemap has been crawled which is good and I will check to see if it got consumed appropriately.   One really good oportunity for you is both getting your site added to the local sites on the search engines as a local listing (think the SERPS version of Yellow Pages) and think about starting a blog.  The frequency of the content in a blog and the possiblity of attracting even better links would make a difference. 

     

    Jeremiah

    Monday, January 14, 2008 8:06 PM

All replies

  •  

    The MSN Bot has indexed over 400 pages in the last day or so, I can actually verify it's pulling pages multiple levels deep successfully, it's just not ever putting them into the index.


    I can tell from the entry pages it appears to be coming in from high profile pages we're on that likewise aren't showing up as backlinks.  I really think something is still amiss and it has to do with our name.

     

    Any ideas Jeremiah?

    Sunday, January 6, 2008 1:14 AM
  •  

    Jeremiah,


    The MSN Bot continues to aggressively index the site today marking the fourth day and close to 500 pages. It's gone through a large percentage of the site this time.   However, will it return any results to the index and if the answer is no will that finally convince the MS people that the problem is on your end?

    There are three types of sites post new algo:

     

    1.  Sites with no trouble

    2.  Sites only found by typing site:www.com, many of which once previously ranked well on MSN

    3.  Sites not found even with site:

     

    Note I see today for the first time in a long time the Hilton site no longer shows up as number one when a search is entered for our name.  It's still 3 or 4 down and the site is not returned yet, but is this progress?

     

    Sunday, January 6, 2008 1:51 PM
  • Jeremiah,

     

    The MSN Bot is continuing to crawl ~50 pages per day and has done so for 4+ straight days which is a first.  It's also deep crawling as it tossed a 401 error on a page way deep in the site.   That's all good news but yet there are still no search results as of this morning.

     

    Is there any chance of getting an answer?  We seem to be in a position unique from others.  We are getting hundreds of pages hit/viewed by MSN per day with success codes being returned in the logs we just can't get any SERP results.

     

     

     

    Monday, January 7, 2008 7:45 PM
  • It'll take time to get ranked. I have found that it'll take a couple of crawls to finally get ranked and in the serps.

    Hey, be thankful you are getting crawled to that extent, many of us are still waiting for the bot to show it's face.

    Good luck,

    Fish Texas

     

    Monday, January 7, 2008 8:50 PM
  •  

    The site's been up for about 24 months and active for 1/2 a year or more.  If it doesn't appear in the next week or two it's never going to.  Judging from all the responses on what I'm sure is 1/1000000000000th of the actual people experiencing the problem I suspect MS is now aware of the issues.

     

    I still believe it's either a perceived spam issue because of the name or just an incapability of the new algorithm to handle plain english. 

     

    The bot continues to index even this morning, but no change in SERPs an the cache hasn't been updated since 11/27.

    Tuesday, January 8, 2008 6:22 AM
  • 60+ pages being indexed per day by MSN non-stop for a week, no change in the index.  Each day it seems to bounce the highly irrelevant terms down a bit, but the actual domain www.promomanagers.com is nowhere to be found.

     

     

     

    Thursday, January 10, 2008 6:26 AM
  • My problem along with PromoManager's seem to  mirror, along with many others, my own. I don't usually resort to forums but  I've been battling with this ranking problem that I feel I must vent my spleen or bust. I don't believe It will do any good because the Livesearch people seem to have their collective heads stuck up a dark place. Anyway here's my story.

    For 2 years my website*www.jefffservice.co.uk* has appeared on MSN SERP's pages in the first 5 positions for my keyphrases. Now, since I first noticed it around the beginning of August 2007, but I believe it happened before then about two months earlier in June, and then nothing appearing in SERP's at all. My site appeared not to be indexed anymore. I re-submitted it, twice, still nothing. The msnbot was crawling my site so why is it not indexing?

     I produced a sitemap.xml file, which in the past I thought unnecessary since my site was featuring very well without it in all the Search Engines including Livesearch. This, I thought, if its a indexing problem this should improve the chances of being indexed. No improvement.

    My robots.txt file included access to all crawlers, but did inhibit image search in Google and MSN.

    I converted, at the beginning of 2007, from XHTML 4.1 Transitional to Web Standards XHTML/CSS Strict 1.0 and duly validated it via the W3C website validation service.

    Submitted my site to Livesearch Webmaster Center/Webmaster Tools(beta) and in the Summary page found 2 pages out of the 8 had indeed been indexed but the last crawl was dated 2 months ago. Also the indexed pages were two minor pages and none of my important ones, which of cause, included my index page.

    Sent countless e-mails to Livesearch Support team basically telling me all the things that I have already done. Including to check my coding is inline with their Webmaster guidelines. I  was eventually escalated to a higher level Search Support  (dizzy heights). Only to be told my site is not blocked and he has re-submitted it. (since he has done that one more page has been indexed along with my sitemap.xml which still leaves five more that are not and I'm still not appearing within the fist 100 pages of SERP's) He also included links to this forum site, which on going through most of the problems that are posted here seems to all mirror the exact same problem that I have.

    The replies to these posts was basically to suggest all the problems was down to sloppy coding of our sites and the suggestion that they were always trying out new algorithms and eventually everything would turn out sweetness and light in the end.
     
    Question - If a site's coding has been validated by the W3C, which forgive me if I'm wrong here is something of an authority on the subject of correct coding, is there some very strange algorithms going on in their search engine?.
    Question - So if a sites not blocked (which mine isn't because MSN Support have said it isn't)) and the coding is OK (which I regard mine as OK see above) the robots.txt file (which MSN Support says is OK and which apparently Livesearch seems to rely on to index pages). My sitemap.xml file has been crawled (because my site logs have registered that it has) So who has the problem?


    Sunday, January 13, 2008 12:35 PM
  •  

    Sadly the vanishing of Microsoft support coincided with the appearance of so many valid sites with identical trouble here on the forum.  One has to consider hundreds of thousands of machines were purchased last month with live search installed and despite this they still lost market share yet again.  It isn't just us noticing, heck on a new machine we got I used the default live search to try to find every day stuff and most of the results were so irrelevant I quickly switched.

     

    The lack of response from Microsoft anti-spam etc is a total disgrace.  It's an embarassment really that such a large company can't get a response to customers within a few weeks or months.  Microsoft makes some terrific products and I've always thought their support was excellent, what is happening with this live search fiasco is just jaw dropping.

     

    Does anyone at Microsoft take responsibility for what's happening and have an actual answer to why your bot crawls 700 pages but indexes none?  Can someone take actual responsibility for this instead of Global sending us here where we get no answers?

    Monday, January 14, 2008 2:32 PM
  •  PromoManagers.com wrote:

    Does anyone at Microsoft take responsibility for what's happening and have an actual answer to why your bot crawls 700 pages but indexes none?  Can someone take actual responsibility for this instead of Global sending us here where we get no answers?

     

    As stated in the past there is a limit to the size of our index and while we would love to index every page on the web and would love to index every site.  We currently cannot guarentee that each site will be indexed and we don't guarentee coverage to any site. 

     

    We are growing our index all the time. We quadrupled our index size in the last year and hopefully we will continue to grow the same way next year.     So why does it crawl 700 pages and index none.  Because we have to select what we think are the best pages.  Not everyone will agree with the design of the selection and ranking algorithm and we are always working on making selection better.

     

    Jeremiah

    Monday, January 14, 2008 7:37 PM
  •  Fergi wrote:


    Question - If a site's coding has been validated by the W3C, which forgive me if I'm wrong here is something of an authority on the subject of correct coding, is there some very strange algorithms going on in their search engine?.
    Question - So if a sites not blocked (which mine isn't because MSN Support have said it isn't)) and the coding is OK (which I regard mine as OK see above) the robots.txt file (which MSN Support says is OK and which apparently Livesearch seems to rely on to index pages). My sitemap.xml file has been crawled (because my site logs have registered that it has) So who has the problem?


     

    Fergi,

     

    Let me answer the qustions first if possible.   Your code will very rarely have an impact on rank unless you are using Flash, Silverlight, or AJAX like scripting without providing an alternate method of navigation.  Design of a site might have an impact, but I looked at your site and it appears fine.

     

    Your second question is the "question of the ages"  In my mind you are asking "I am doing everything right, but I am not ranking?"

     

    So I took a look at some of the site linking to you...  You can see these in our webmaster tools or in any of the other search engine tools.

     

    Here are the top ten at the moment I looked at your site:

    www.turnpike.net/directory.html?showcat+home+280
    www.turnpike.net/directory.phtml?showcat+home+240 
    metro.turnpike.net/directory.phtml?showcat+home+22
    metro.turnpike.net/directory.phtml?showcat+home+24
    www.etkal.com/directory/Homemaking/Homemaking-1.ht
    www.zabalook.com/Home/Consumers.php?a=19 - SP
    www.excellentweb.info/directory/Consumer-Informati
    www.esgsite.com/index.php?c=391&p=7 
    www.cocovida.co.uk/links/rss.php?c=9
    www.fatinfo.com/home/Tumble-Dryer-repairs-in-Kent

     

    Following these links they look mostly like directorys which don't really have any authoritative rank in the case of a couple actually appear to be link farms.  These are not the kind of links that will help you build rank infact they may hurt you.

     

    Quality links can be directories, but the content and context of the directory needs to be really good. (see industry association below). If you still want to build links with directories, I recommend being very choosy. You could looking into the following link building methods.

    • Join a reputable industry association – often times they will list their members and provide links to their websites. Examples could be a local rotary club, or a professional association like the American Medical Association.
    • Get involved with your community – participating with your community through blogs, forums and other online resources may give you legitimate reasons to provide links to your site.
    • Press Releases – if your company has a significant event, consider doing a press release through a site like http://prweb.com.
    • Suppliers and partners – ask your business partners if they would add a section to their website describing your partnership, with a link to your website. Or, if your suppliers have a website, perhaps they have a page where they recommend local distributers of their products.

    The reality is that you are in the index but your site is not ranking well.   Your sitemap has been crawled which is good and I will check to see if it got consumed appropriately.   One really good oportunity for you is both getting your site added to the local sites on the search engines as a local listing (think the SERPS version of Yellow Pages) and think about starting a blog.  The frequency of the content in a blog and the possiblity of attracting even better links would make a difference. 

     

    Jeremiah

    Monday, January 14, 2008 8:06 PM
  •  

    Jeremiah can you answer this question this time:

     

    1.  Why does your search engine think a hotel site who's only reference to promomanagers is term promo=managers in the url outrank a site with that exact name?

     

    Year over year you are down 28% per Hitwise and you're still bleeding almost 1% a month (despite all the new computers that ship with live as their default search), whatever you've added for URL's it's pretty clear most don't find it useful. 

     

    It's astounding to hear the largest software company on the planet has a limit on the size of the data it can store. 

     

     

    Fergi, dont't waste your time or money on those suggestions.  Being listed with industry assocations doesn't help, press releases haven't helped, major magazine articles haven't helped.  Those things only work if MSN actually re-indexes the pages you end up on.   As of today MSN none of those suggestions have worked, if you do it with the hope of getting reindexed you're just wasting your money.  In short you could spend a few hundred or thousands of dollars and you'll still be at square one.

     

    This all falls on deaf ears as they say because it's much easier to blame the customer/company.

     

    Monday, January 14, 2008 8:08 PM
  •  

    Still hoping to get an answer on keyword stuffing of irrelevant terms in a URL outranking underlying sites.

     

     

    Wednesday, January 16, 2008 3:44 PM
  • Hi Jeremiah

     I think you touched on the real issue here and that is your link detecting algorithms (of course I'm guessing but in this game that's all you ever do) are now insisting on popularity and the theory being a site is only good if it has lots of links going to it. This is Google all over again, Larry Page and Sergey Brin remember them?!. Now my site has been going for 3 years and when I first submitted my site to them at that time for a few months I was ranking very well. Then they changed their algorithms, there were several, but one I know was the aging algorithm, which I learned latter, was if your site was new it was held back in the rankings for at least 9 months or more and of course they had their page ranking which mainly was assessing your inbound links and the relevance of them.

    Now the question is, with Google I was crawled and indexed but I was actually appearing in SERP's but down in page 70 somewhere, which of course is no good to man or beast. Then, low and behold, after 9 months, not straight away, but gradually I crept up the SERP's pages to then reached first or the top of the second page for my most competitive
    keyphrase. BUT this happened in spite of the fact I had very few inbound links. My theory is that if I had lots of relevant links I would have ranked faster but since the site its self was good it got there but slowly, just a thought. Now in your response to your list of my Directory inbound links to my site, these links have only appeared SINCE December last year (I commissioned a Company who hand-submitted and not auto-submitted to Directories only) but the problem with Livesearch started in August (or before) in 2007. So this problem I have is nothing to do with these links. Perhaps there may be a few iffy ones, I'll have to look into that, but its not the problem we have here.

    The point I'm trying to get at is, is that with Livesearch I'm not appearing anywhere within the first 100 pages and haven't been since mid 2007. So, bearing in mind and as far as I'm concerned, my on-page optimization is fine and its the off-page that I might have a problem with. I take that on board. So if its a inbound link issue, and I  realize that every other Search Engine is different to their approach to Search, I seem fine as far as their are concerned and also bearing in mind it is now in its 6th month I have been out of the rankings in Livesearch (but could be longer) of this problem starting. Why is it taking so long  to appear anywhere in Livesearch SERP's?  I'm not even up to page 70!

    Question - Do you have an aging algorithm? In 9 months will I suddenly appear? Surely I've not been sandboxed all this time because of links, because that is the only conclusion I can come to. Because if it is, your algorithms are either too critical or too slow for good sites to work their way up.

    As for blogs, how much about washing machines (or dishwashers or tumble dryers) can you talk about. In the business I'm in my customers don't want to or haven't got the time to muck about with my ramblings. They want their machine repaired. Like yesterday!  As for links, its a funny thing, in my business there are no associations. All the people and suppliers I'm in contact with there're not interested in search engine ranking because their websites function purely for their customers and are used only to look up and order parts on. They are password protected so they are part of the Invisible Web. They have no ranking so even if I had links from them it probably wouldn't help me anyway.
     
    If a site is good as it stands on its own merit and offers a service that when customers are searching for 'washing machine repair in kent' it should come up with a well designed well on-page optimised site because the Search Engine should be good enough by now to distinguish good and bad sites without the need for blogs or links. Because if I have to optain links  and run a blog I'm doing it to please the Livesearch Search Engine rather than for the benefit of my customers.
     
    Question - How many (relevant links) would I have to get to make a difference to your Search Engine? Hundreds?

    Question - If I started a Blog to-morrow would it begin to advance my site up the rankings in Livesearch?

    Fergi

    Wednesday, January 16, 2008 8:24 PM
  • Hi Fergie,

     

    Funny thing is that my site IS a blog... and it's disappeared from Live Search. LOL.

     

    I've written over 150 entries on it. I regularly comment on other blogs, other people write my blogs up on their blogs etc... I've had a huge response from people all over the world... except from Live Search.

     

    No disrespect to the guys working here who are trying valiantly to quell the fire, but....

     

    Reviewing this forum I think that this is a huge stuff up on the part of Live Search whom I formerly thought to be the number 2 search engine.

     

    Fortunately over 85% of Australians (my market) use Google. Less than 4% use MSN or Yahoo, so I'm not missing out on much. They certainly won't be making any gains with this rubbish.

     

    Unfortunately, it seems that they've developed a major size issue. It's too small. LOL.

     

    They're throwing out useful sites left right and centre! IT is a shame. Let's hope they can turn it around before this becomes a major PR disaster.

     

    Oh well, I have some fodder for my blog tomorrow..

    Thursday, January 24, 2008 2:21 PM
  •  I,ve had no answer to my questions in my post of the 16/01/2008 so I'll repeat myself.

    If a site is good as it stands on its own merit and offers a service that customers are searching for as in 'washing machine repair in kent' it should come up with a site that if designed well and that is relavant to the search query entered then that should be good enough, because the Search Engine should be good enough by now to distinguish good and bad sites without the need for blogs or links. Some links certainly but there has to be a balance and Livesearch hasn't, at the moment, got it. If I have to optain links and run a blog, as is what is suggested, I'm doing it to please the Livesearch Search Engine rather than for the benefit of my customers. I'll repeat my questions

    Question - Do you have an aging algorithm? In 9 months will I suddenly appear?

    Question - How many (relevant links) would I have to get to make a difference to your Search Engine? Hundreds?

    Question - If I started a Blog to-morrow would it begin to advance my site up the rankings in Livesearch?

    Fergi



    Friday, January 25, 2008 10:05 PM
  •  

    Taking Jeremiah's information we've gone out of our way to get listed in high ranking directories. We've made sure the link structure is strong and have gotten the message out as much as we can on quality websites.  It appears to be having some effect on the number of times MSN is visting the site, the BOT visits daily but still nothing is getting indexed or updated. 


    What's strange is reviewing the bots steps today it does pull the robots file but then ignores it.  We played around today trying to keep it off two pages it ended up visiting anyway, promotional coolers and promotional writing pads.  The MSNBot follows no rhyme or reason and bounces all over the hompage.  It is still working off the old index as it repeatedly checks products that were once on the homepage but are now gone.  It does not deep crawl and does not crawl related items with direct links on pages.  It's visited the Port Authority Competitor Jacket page a bunch of times, but never kicks off to the related items listed.

     

    What we're trying to figure out is whether or not it's actually using the sitemap files, the answer appears to be no as it has not yet pulled a new copy and we've checked every single day.  Either the pinging service is broken, or is seriously lagged.

     

    I'll report what we see every few days.  Maybe we're missing something and it's kicking to certain pages for a reason, or maybe there's a hint here somewhere that MS will pick up on.

    Saturday, January 26, 2008 3:31 AM