About the Robots.txt RRS feed

  • Question

  • Hi, I have creat my  robots.txt file like this :


    # Robots.txt file from http://www.dfdsolar.com
    # All robots will spider the domain
    User-agent: *


    And i use the Live Robots.txt validation tool to evaluate my Robots.txt file, the result showed like this:

    Warning: 'sitemap' - tag isn't specified.

    What that mean?

    And how can i correct my robots.txt



    Thursday, January 8, 2009 4:55 AM


  • Hi Land,


    Your robots.txt is fine. That warning was referencing the fact that you do not have a sitemap location specified.

    (sitemap: http://www.example/sitemap.xml) I suggest looking at robotstxt.org or sitemaps.org for more information.





    Tuesday, January 13, 2009 6:24 PM