locked
Site largely ignored by robot RRS feed

  • Question

  • I registered this site http://www.kus.com.au with Webmasters Tools some months ago. It has a sitemap which was declared when the site was registered. In addition to the sitemap, it uses simple anchor tags for its internal navigation. It is linked from other sites (confirmed by Webmaster Tools on the Backlinks page). As far as I can tell the home page has been visited only once by MSNBot in the last couple of months. No other pages have been visited. The site has existed for at least 2 years.

    The site is listed in Yahoo and Google. It is built with clean html. It does not use any "black" techniques. Yet the site has not been listed in the Live index (confirmed  bu the Webmaster Tools and by a site:kus.com.au query in Live).

    Can anyone suggest why Live Search doesn't like the site or how much more patience I need?

    Thanks.
    Friday, November 28, 2008 11:48 PM

Answers

  • Rebaf,

    I looked at your site and the site looks good, there are no technical issues. The one thing you could do is generate more backlinks to the site. Today there are not many high quality links to the site.  The truth is that w
    hile Live Search crawls billions of webpages, not every webpage is indexed.  Generating some additional links and adding additional content could help.

    One way to generate additional traffic and links is to start a blog. Given your unique industry this might be something worth a try or perhaps you could getting links from a industry association you belong to.

    I would try a few link building techniques and then get back to us and let us know how it goes.

    Jeremiah



    Sunday, November 30, 2008 4:52 AM
  • Forums is something worth trying to.  I had not thought of that as an answer. Most webmsters can do themselves a lot of good by trying creative solutions to generating traffic. This is not a bad idea.

    Jeremiah
    Sunday, November 30, 2008 5:28 AM

All replies

  • Rebaf,

    I looked at your site and the site looks good, there are no technical issues. The one thing you could do is generate more backlinks to the site. Today there are not many high quality links to the site.  The truth is that w
    hile Live Search crawls billions of webpages, not every webpage is indexed.  Generating some additional links and adding additional content could help.

    One way to generate additional traffic and links is to start a blog. Given your unique industry this might be something worth a try or perhaps you could getting links from a industry association you belong to.

    I would try a few link building techniques and then get back to us and let us know how it goes.

    Jeremiah



    Sunday, November 30, 2008 4:52 AM
  • Forums is something worth trying to.  I had not thought of that as an answer. Most webmsters can do themselves a lot of good by trying creative solutions to generating traffic. This is not a bad idea.

    Jeremiah
    Sunday, November 30, 2008 5:28 AM
  • Jeremiah

    Thanks for confirming there are no technical difficulties with the site.

    I'm pleased to report that the 'bot has started to visit since the start of December. My patience was further rewarded by seeing the site appear in the index. This was confimed by this search - http://search.live.com/results.aspx?q=site:kus.com.au.

    However, something is still not right. The 4 pages listed are all shown with .htm extensions but the real pages at the site are accessible with the .html extension. The names of the pages are all valid except for the missing "l" on the page extension. Can you shed any light on this for me?

    Thanks.


    Tuesday, December 2, 2008 10:30 PM
  • It may be purely coincidence but I implemented the suggestion to do 301 redirects for the non-existant pages to the real versions a couple of days into December. Now these pages have dropped from the webmaster Summary listing and only 3 pages are shown in the index in response to a site: query.

    The robot has continuted to visit during December. It has requested robots.txt once. One request was for one non-existant page. The remaining requests have all been for the home page. The user agent for all but one of the robot visits included "msnbot-webmaster".

    Yeah yeah - patience. I get that. What I'm missing is the logic. There doesn't seem to be much rhyme or reason to what's happening here. Anybody want to offer an opinon?
    Wednesday, December 17, 2008 4:23 AM
  • How much patience is enough?

    It's now several weeks later and the robot has stopped visiting, in fact it hasn't dropped by to nibble so far this year. The non-existant pages have dropped from the index and there is still no sign of the real pages in the index. The site has not had any downtime recently and has never used any black-hat techniques. I'm not expecting to rank seriously for any keywords but I would like the site found for a site:kus.com.au query. It's about 3 months since I added the site to my MSN Webmaster facility and made the sitemap available.

    An earlier suggestion from Jeremiah was to add a Forum. Maybe I'm just missing the point. I can understand that a Forum may be a way to improve the overall ranking but since when is adding a Forum to a site a means to get the site included in the index?

    All comments / suggestions are appreciated.

    Sunday, January 11, 2009 7:12 AM
  • Can you clarify what you mean by "garlic"?
    Thursday, January 15, 2009 8:32 AM