Sign in
Microsoft.com
 
Microsoft
 
 
 

locked
Robots.txt says DisAllow but I am still listed in MSN??? RRS feed

 > 

  • Question

  • Question
    Sign in to vote
    0
    Sign in to vote
    Hello All:

    I have a robots.txt file on my server with the following

    User-agent: *
    Disallow: /

    However, in my webmaster tools for this site, under blocked is says no. Whats more, it comes up if you search out this site.

    This site is a development site and should not be viewable by the public.

    Google and Yahoo seem to work, just MSN is indexing it for some reason.

    Any help or advice woult be great help,

    Paul
    Wednesday, February 20, 2008 8:55 PM

Answers

  • Question
    Sign in to vote
    0
    Sign in to vote
    It's been posted before that the bot is not always paying attention to the robots file.  We've run a trial with two files that still got indexed even when our robot file tried to exclude MSN.  Like others have said, it took us months to get our promos site indexed, it's funny to see you get it without wanting it. 

     

    Friday, February 22, 2008 1:07 AM

All replies

  • Question
    Sign in to vote
    0
    Sign in to vote

    this is the first time i have heard of someone who didnt want their site to be seen and it still shows up...most of the sites cant even get indexed.

    Thursday, February 21, 2008 10:39 PM
  • Question
    Sign in to vote
    0
    Sign in to vote
    It's been posted before that the bot is not always paying attention to the robots file.  We've run a trial with two files that still got indexed even when our robot file tried to exclude MSN.  Like others have said, it took us months to get our promos site indexed, it's funny to see you get it without wanting it. 

     

    Friday, February 22, 2008 1:07 AM