locked
site seen without www - canonicalisation issue? RRS feed

  • Question

  • We are geographically targeted to the UK as we are a relatively small community website in the UK.

    It does appear that MSN is seeing our domain http://www.crowland-links.com WITHOUT the www. Our sitemap used is WITH the www and we use htaccess to provide a 301 should anyone search the domain WITHOUT www.

    Looking at the ONLY page indexed by MSN (7 days after sitemap submission), it shows http://crowland-links.com and NOT the prefered version which uses www.

    A Live Search for keyword "crowland community" which is often number 1 in Google, MSN shows the home page WITHOUT www and what's even worse, shows the description as shown in DMOZ and not the site's Meta description.

    There does appear to be some real issues with the way MSN crawls, as the other principal spiders have no problem (Google has indexed about 400 of our pages and we are PR 3 - not bad for a town community website).

    First, how come MSN has indexed us WITHOUT the www?

    Second, why has MSN only indexed ONE page (and not with the correct
    canonicalisation) ?

    Third, why is MSN using the DMOZ listing (which is www.crowland-links.com) but DROPPED the www.

    And last, why is MSN using the DMOZ entry at all?

    Using a different WMT is often a learning curve - I would appreciate some help with this one please

    Thursday, October 30, 2008 4:43 PM

Answers

  • Hi Profaders,

     

    Currently I show 21 pages indexed with the www version of your site. I also see that we are using your meta description now.  If you need anything else, please let me know.

     

    Friday, December 5, 2008 4:11 PM
  • Profaders,

     

    Looking the issue, there are a number of backlinks that point to the non-WWW URL.  Given that you recently made the change, the homepage will probably be listed without the WWW for a while. The best way for you to speed this up is to both build additional links to the canonical URL or track down and have those linking to you without the www change it.   The 301 will over time over ride this, but if you are looking for more immeadiate results you may want to try this.

     

    Thanks

     

    Jeremiah Andrick

    Tuesday, December 16, 2008 5:24 PM

All replies

  • Hi,

     

    I  would be glad to help you. With regards to the canonicalization issue, how long have you had the redirect in place?

     

    DMOZ:

     

    We pull descriptions from various sources such as from page content, Meta descriptions and from DMOZ entries as does Google. This happens often enough that there is a specific meta tag to prevent using the DMOZ descriptions. If you don't want the DMOZ entry, I suggest placing <meta name="robots" content="NOODP"> Meta tag in your header which will prevent robots from using the DMOZ entry.

     

    There are many reasons why we index only a certain number of pages including our index size, site ranking, site architecture issues, etc. Have you used our webmaster tools to verify there are no crawling issues? You might also want to review our crawling and ranking blog post.

     

    Thanks!

     

     

    Brett

     

     

    Friday, October 31, 2008 5:45 PM
  •  Brett Yount wrote:

    I  would be glad to help you. With regards to the canonicalization issue, how long have you had the redirect in place?


    Thank you Brett - nice to know that you are keen to help.


    The htaccess redirect has been in place since 3rd August 2008. Up and until we added our sitemap in the last 14 days, we were NOT showing in the MSN SERPS - now we are - but showing without www.


     Brett Yount wrote:

    DMOZ:

     

    We pull descriptions from various sources such as from page content, Meta descriptions and from DMOZ entries as does Google. This happens often enough that there is a specific meta tag to prevent using the DMOZ descriptions. If you don't want the DMOZ entry, I suggest placing <meta name="robots" content="NOODP"> Meta tag in your header which will prevent robots from using the DMOZ entry.


    Yes, I'm familiar with NOODP. And if I had similar issues with other SEs then yes I would utilise this - but I have no such issues.


    As you say, Google do not pickup on the ODP descriptions if there is a valid Meta Description. Our index page Meta Description has not changed for almost 10 weeks - and change, I mean a word or two, nothing more. So if Google (and all the other SEs think it's valid and by the definitions you imply) why does MSN not see it as valid? It is a valid Meta (check it out) so why not use it ? We had valid META Descriptions a good time BEFORE our DMOZ listing.


     Brett Yount wrote:

    There are many reasons why we index only a certain number of pages including our index size, site ranking, site architecture issues, etc. Have you used our webmaster tools to verify there are no crawling issues? You might also want to review our crawling and ranking blog post.


    ALL aspects listed in this blog are complied with (the same as Google) - if not, then we would NOT be listed in Google, given PR3 and high in SERPS, plus showing individual Meta Descriptions, as we individually craft ALL our Metas using professional SEO practice.


    I can confirm the msn WMT shows NO site crawling issues (nor do any other competitor SEs show crawling issues).


    Something not quite right with all of this, don't you think?

    Friday, October 31, 2008 7:38 PM
  •  Brett Yount wrote:

    Hi,

     

    I  would be glad to help you...

     



    No reply to my last post?

    Then I can only assume there are known issues with MSN crawls, as to index in the way MSN has done here simply defies gravity. Totally ignores the XML site map, and also the robots.txt instruction for Sitemap: Simply goes to ODP and grabs that - but then still gets it wrong by indexing without the www.

    MSN WMT shows 2 pages indexed: one is the sitemap XML and the other doesn't exist in reality as it's without the www, and if it were not for my my htacess redirect it wouldn't find that either.


    Thursday, November 6, 2008 2:25 PM
  • Hi Profaders,

     

    Currently I show 21 pages indexed with the www version of your site. I also see that we are using your meta description now.  If you need anything else, please let me know.

     

    Friday, December 5, 2008 4:11 PM
  •  Brett Yount wrote:

    Hi Profaders,

     

    Currently I show 21 pages indexed with the www version of your site. I also see that we are using your meta description now.  If you need anything else, please let me know.

     



    Thank you, but my principal question STILL goes unanswered - why is my index page STILL showing without www.  ?

    I give up!
    Monday, December 15, 2008 12:25 PM
  • Hi Profaders,

     

     I'm still researching this issue with devs. I'm sorry if I gave you indication otherwise. I just wanted to report a positive change.  

     

    Monday, December 15, 2008 7:50 PM
  • Profaders,

     

    Looking the issue, there are a number of backlinks that point to the non-WWW URL.  Given that you recently made the change, the homepage will probably be listed without the WWW for a while. The best way for you to speed this up is to both build additional links to the canonical URL or track down and have those linking to you without the www change it.   The 301 will over time over ride this, but if you are looking for more immeadiate results you may want to try this.

     

    Thanks

     

    Jeremiah Andrick

    Tuesday, December 16, 2008 5:24 PM
  •  Jeremiah Andrick - Microsoft wrote:

    Profaders,

     

    Looking the issue, there are a number of backlinks that point to the non-WWW URL.  Given that you recently made the change, the homepage will probably be listed without the WWW for a while. The best way for you to speed this up is to both build additional links to the canonical URL or track down and have those linking to you without the www change it.   The 301 will over time over ride this, but if you are looking for more immeadiate results you may want to try this.



    Thank you Jeramiah and Brett.

    You say a "number" of backlinks?

    I don't know where MSN has found these - perhaps this is the real problem.
    Out of 41 links, the only NON www version backlink that MSN shows me is to my own index page! (the very issue) - plus TWO which are directing to http://dmoz... but my dmoz listings in every directory all show the correct canonicalisation. It looks like I'm locked into a self-fulfilling MSN loop here!

    I have 69 links showing in Google - NONE are WITHOUT www. so I have no idea where you or your bot are getting these from. Further, 4 months is a long time in SERPs indexing - so why does it take so long for MSN indexing to refresh?

    You can't simply go and "build additional" links (besides, all links ARE www. apart from those which I make comment). There are such things as "natural" links. Gone are the days of link exchange schemes and paid links - look at the spamming mess the net is in because of all that rubbish. Looks like MSN is not quite up to date with things in the indexing world.

    Smile
    Sunday, December 21, 2008 12:37 PM