Is there a way to limit the amount of data MSNBOT can download from my website daily?
I get around 1K hits a day and with my pages being 500KB this is somewhat of a problem.
What is even more annoying is that pages hardly ever change yet they are being recrawled and recrawled...
Please help!