Forum Moderators: goodroi
how can i block only the file with ref=example.com from crawling by google....
example.com is an malware domain - and i don't know why my Server response with a 304 - please - i have to disallow Google to crawl that site with that Malware Parameter...
Disallow: example.com
does this work ?
thanks in advance...
There is nothing you can do about this with a robots.txt file.
The 304 response means that the content has not changed since last requested.
[edited by: Samizdata at 9:10 pm (utc) on Mar. 16, 2008]
file.html is not the same like file.html?ref=example.com
i do not have mod_rewrite :( - please how is the syntax to prevent Googlebot from spidering that fake query?
i need disallow: ref=
or what ? i do not use querys with ?ref= - so it must be easy to disallow Googlebot to spider that file with this query?