Welcome to WebmasterWorld Guest from

Forum Moderators: Robert Charlton & goodroi

Message Too Old, No Replies

Stopping url being indexed?

Not with robots.txt

11:32 am on Nov 29, 2005 (gmt 0)

Junior Member

10+ Year Member

joined:May 9, 2005
votes: 0

Is there any other way to stop a URL being indexed other than use robots.txt? We have a statistics script that loads at the end of the html but robots have been following it and getting caught in an endless loop.

I have posted an example of the code before,

<img src="http://www.mysite.fr/stats/getStats.cfm?passed_sessionkey=1132945495-14913556&passed_scriptname=&passed_jobref=&passed_http_request_method=GET&passed_path_translated=D:\eFC\index.cfm&passed_script_name=/index.cfm&passed_query_string=&passed_HTTP_referer=&partnerId=003&passed_uniqueUser=C2430036-1321-B04D-82CCC92A61FA1FAC" width="0" height="0" style="display:none">

Anyone know a way in html that could be used to stop spiders from indexing this link - notice the link is dynamic and will change each visit.

Any help apprecaited.

7:07 pm on Dec 1, 2005 (gmt 0)

New User

10+ Year Member

joined:Nov 18, 2005
votes: 0

Ya, just add


and if your problem is also with google, try


in your <HEAD></HEAD>.

That should fix you up.

- Shaun

6:52 pm on Dec 2, 2005 (gmt 0)

Junior Member

10+ Year Member

joined:Aug 11, 2004
votes: 0

I'm just thinking out loud here so somebody please correct me if i'm wrong, but with your link


what if because the URL is different you're not properly definining the file name in robots.txt. Perhaps in robotos.txt deny all for the directory so that everything within it is banned. Also adding the no index meta tag to the page too sounds good.

Also still, if it's a bad bot.. try .htaccess in that directory and ban by IP.

I hope this helps.