locked
Only One Page Indexed RRS feed

  • Question

  • For both my blogger.com blogs, only one page has been indexed in site summary. I have almost 100 posts on them. I really want to use the live search on these sites but it can't give back any meaningful results because my pages aren't indexed.

     

    I've set up my sitemap as such (correct if I am wrong):

    Sitemap web address: mywebsitename.blogspot.com/rss.xml?max-results=500&start-index=1

    I've done the ping as well.

     

    I realize the crawler really looks at meta tags but on a blog you can't really do that.

    Need Help!

    Thursday, April 10, 2008 3:56 PM

Answers

  • Hi,

     

    There are many reasons why your site is not getting indexed completely. As stated in the past, there is a limit to the size of our index and while we would love to index every page on the web and would love to index every site, we currently cannot guarantee that each site will be indexed and we don't guarantee coverage to any site. 

    We are growing our index all the time. We quadrupled our index size in the last year and hopefully we will continue to grow the same way next year. So why does it crawl so many pages and index none?  Because we have to select what we think are the best pages.  Not everyone will agree with the design of the selection and ranking algorithm, however, we are always working on making selection better.

     

    Also, although we have devs working on the solution, we are still experiencing issues with 301 redirects.

     

     

    Brett

    Tuesday, April 22, 2008 5:48 PM

All replies

  • I have the same exact problem myself, I have totally restructured my site this month but I was only crawled on 02/08. It has no chance to even try to crawl my new content so it can index it. I know that I had my non-www versions of my site indexed at one point, so I 301 redirected non-www traffic to the regular www.domain.com. In google this works perfect and I have no duplicate content showing up now.

    I have a feeling that it is only trying to index my non-www version and since I use a 301 redirect there it can't go any further. I read that live can't handle 301's properly, but If I take it out I will have duplicate content issues on yahoo and google.

    It's just crazy that I only have my 1 root page in the index, the links are all easily followed by other engines(no javascript or other nonsense).

    Maybe you are also doing a 301 redirect to fix canonicalization issues? Most big sites do this exact thing as well and I am noticing that even they are having idexing issues with Live search.

    Well it's just an idea, might have nothing to do with this.

    -Tim


    Thursday, April 17, 2008 12:30 PM
  • After more reading, A LOT of people are having the same problem as us. I contacted the live search team but got the automated response, I might try explaining more but first I think i'll remove the 301 redirect. Just as an experiment I will risk getting some duplicate content issues on Google, this shouldn't do anything to my ranking anyway. Well if I get indexed I think we'll know that this is part of the problem.

    My site is in no way a spam site nor am I doing any sort of SEO trickery, but as an ex employee of a SEO firm I know what needs to be done to follow the webmaster guidlines. I have valid pages and useful content on every page as well, not sure why it's blocked.

    Will post my results on this forum.

    -Tim
    Thursday, April 17, 2008 1:59 PM
  • Hi,

     

    There are many reasons why your site is not getting indexed completely. As stated in the past, there is a limit to the size of our index and while we would love to index every page on the web and would love to index every site, we currently cannot guarantee that each site will be indexed and we don't guarantee coverage to any site. 

    We are growing our index all the time. We quadrupled our index size in the last year and hopefully we will continue to grow the same way next year. So why does it crawl so many pages and index none?  Because we have to select what we think are the best pages.  Not everyone will agree with the design of the selection and ranking algorithm, however, we are always working on making selection better.

     

    Also, although we have devs working on the solution, we are still experiencing issues with 301 redirects.

     

     

    Brett

    Tuesday, April 22, 2008 5:48 PM