locked
Validation Problem of Robots .txt RRS feed

  • Question

  • Hi all I am facing this problem while validation my robots.txt in MSN webmaster Message appear as

    First my Robbots.txt is

    # Alexa
    User-agent: ia_archiver
    Disallow:

    # Ask Jeeves
    User-agent: Teoma
    Disallow:

    # Google
    User-agent: googlebot
    Disallow:

    # MSN
    User-agent: msnbot
    Disallow:

    # Yahoo!
    User-agent: Slurp
    Disallow:

    # Abacho
    User-agent: AbachoBOT
    Disallow:

    # Baidu
    User-agent: baiduspider
    Disallow:

    # Fireball
    User-agent: fireball
    Disallow:

    # ObjectsSearch
    User-agent: ObjectsSearch
    Disallow:

    # Szukacz
    User-agent: szukacz
    Disallow:

    # Voila.fr
    User-agent: VoilaBot
    Disallow:

    # Walhello
    User-agent: Appie
    Disallow:

    # Yandex
    User-agent: Yandex
    Disallow:

    # Others
    User-agent: *
    Disallow:

    Validations Results:

    Line #55: Disallow:
    Error: Conflicting disallow tags found.
    **************************************************
    Warning: 'sitemap' - tag isn't specified.
    **************************************************

    Could you please tell me how to solve it.I will be very thankful to you all.


    Monday, June 23, 2008 11:53 AM

Answers

  • What I don't quite get is if you want all spiders to crawl your entire site, why are you going through the trouble to list them out seperately?

     

    Wednesday, July 9, 2008 2:10 AM