Alternative to Wildcard RRS feed

  • Question

  • I have 3000 pages that I don't want indexed, so I put a directive in my robots.txt that tells all crawlers that I don't want any page indexed that includes /cgi-bin/report.cgi*


    Unfortunately I have discovered that MSNBOT doesn't support wildcards. So what is the alternative?


    Thursday, July 3, 2008 12:09 AM


All replies