locked
About the Robots.txt RRS feed

  • Question

  • Hi, I have creat my  robots.txt file like this :

     

    # Robots.txt file from http://www.dfdsolar.com
    # All robots will spider the domain
    User-agent: *
    Disallow:

     

    And i use the Live Robots.txt validation tool to evaluate my Robots.txt file, the result showed like this:

    Warning: 'sitemap' - tag isn't specified.
    **************************************************

    What that mean?

    And how can i correct my robots.txt

     

     

    Thursday, January 8, 2009 4:55 AM

Answers

  • Hi Land,

     

    Your robots.txt is fine. That warning was referencing the fact that you do not have a sitemap location specified.

    (sitemap: http://www.example/sitemap.xml) I suggest looking at robotstxt.org or sitemaps.org for more information.

     

     

    Brett

     

    Tuesday, January 13, 2009 6:24 PM