locked
robots.txt validator error RRS feed

  • Question

  • I'm having problems validating my robots.txt files within 6 websites that I manage and cannot for the life of me see why. The error is:

    Line #1: www.moneymatchmaker.com/robots.txt
    Error: 'user-agent' -  should be the first tag.
    **************************************************
    Warning: 'sitemap' - tag isn't specified.
    **************************************************


    Can anyone through some light on the problem?

    Thanks

    Sparkie
    www.webprosolutions.co.uk
    Tuesday, January 29, 2008 12:30 PM

Answers

  • Mark, it looks like you typed the path to your robots.txt into the validator tool instead of pasting the contents of the file into the tool.
    I took a look at your robots.txt file and it looks like you have two small problems: there are single quotes around the user-agent tag and angled brackets around the sitemap path. I have highlighted bellow the characters you’ll need to remove from your robots.txt.

     

    Code Snippet
    'user-agent': *
    sitemap: <
    http://www.moneymatchmaker.com/sitemap.xml>
    disallow: /media/
    disallow: /error.php

     

     

     

    Thanks,
    Eric Carter

    Tuesday, January 29, 2008 5:55 PM