locked
how long does it take? RRS feed

  • Question

  • I submitted my sitemap back in mid-February. It still hasn't been crawled. The site itself has. How long does it typically take for msn to crawl and update its index?

    David

    Sunday, May 18, 2008 12:43 PM

Answers

  • Hi David

     

    All you need is to point the spiders in the right direction, You don't need all 32000 links and you can use more that one sitemap.

     

    If you create a sitemap.htm file and add say 100 links or if thats two many 50 you then create a sitemap.php file or whatever code your using sitmap.asp

     

    Each link on the htm file tells the php to run a combination of product queries links  example

     

    htm file

    domainname.com/sitemap.php?show_prods=1

     

    PHP file

    Switch (show_prods)

     

      case '1'

          Select pro_id, pro_name FROM products WHERE cat_id = 1 AND cat_id=2

     

    Then run a loop through each result to build your sitemap links

     

              echo  '<a href=http://www.mydomain.com/products_details.php?prodID=' . mysqlresult($i,pro_id) .'>' . mysqlresult(pro_name) .'</a><br />';

     

     

    Andy

    Tuesday, May 20, 2008 8:24 AM

All replies

  •  Hi David

     

    Have any of your pages in your sitemap been indexed?

     

    Andy

     

    Sunday, May 18, 2008 1:07 PM
  • Yes, all the pages accessible by normal links on my site like category headings have been indexed. The individual product pages which are addressed in the sitemap haven't.


    Sunday, May 18, 2008 1:16 PM
  • Hi David

     

    Most of us in this forum are having problems with indexing been dropped or live search will not crawling. You are very lucky your site has been index at all.

     

    There's no time scale, but if your open links are indexed it would be good practise to provide a link say on your homepage to a sitemap.html page with links to all your product link pages.

    This not only helps search engines find all your hidden pages but also gives your readers a quick way of finding items within your site.

     

    Andy

     

    Sunday, May 18, 2008 2:15 PM
  • thanks. I'm a bit hesitant to put a sitemap html page up. My site is all database driven which is why a lot of pages aren't getting indexed without the sitemap. The spiders only take the first page of returns on any given heading and don't follow the numbered page links to continue with the remainder of query returns. The sitemap has over 32,000 links on it which if someone tries to load in a browser would probably cause problems.
    Monday, May 19, 2008 11:54 AM
  • Hi David

     

    All you need is to point the spiders in the right direction, You don't need all 32000 links and you can use more that one sitemap.

     

    If you create a sitemap.htm file and add say 100 links or if thats two many 50 you then create a sitemap.php file or whatever code your using sitmap.asp

     

    Each link on the htm file tells the php to run a combination of product queries links  example

     

    htm file

    domainname.com/sitemap.php?show_prods=1

     

    PHP file

    Switch (show_prods)

     

      case '1'

          Select pro_id, pro_name FROM products WHERE cat_id = 1 AND cat_id=2

     

    Then run a loop through each result to build your sitemap links

     

              echo  '<a href=http://www.mydomain.com/products_details.php?prodID=' . mysqlresult($i,pro_id) .'>' . mysqlresult(pro_name) .'</a><br />';

     

     

    Andy

    Tuesday, May 20, 2008 8:24 AM
  • Hi Andy,

     

    thanks for helping out.

     

    Wednesday, May 21, 2008 8:08 PM
  • thanks. How do I point the spiders to the page? I don't want to just put a link on the site that customers might click on since the returns would be meaningless to them.
    Thursday, May 22, 2008 12:26 PM
  • But the links would'nt be meaningless because they would all be pointing to your products

     

    <a href="products/product_details.php?prod_id=230">Music by holst??????</a>

    Page output

    Music by holst??????

     

    Just simply add a link at the bottom of your homepage or any page that has been indexed. Most people don't look at sitemaps anyway its the same as who reads the terms and conditions / privacy pages?

     

    If you want to get all your products index this is the quickest way. What you are doing is giving all your products their own pages in away once they find a batch of them they will just keep crawilng

     

    Andy

    Thursday, May 22, 2008 12:52 PM