homepage Welcome to WebmasterWorld Guest from
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member
Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt
Forum Library, Charter, Moderators: goodroi

Sitemaps, Meta Data, and robots.txt Forum

Googlebot chasing .php tracking script
is robots.txt enough?

10+ Year Member

Msg#: 273 posted 7:05 pm on Feb 8, 2004 (gmt 0)

I recently added an ROI tracking program to a site and I'm pretty happy with the reports it produces. The following code is placed on the pages to implement the tracking (altered for this post):

<script type="text/javascript" src="/a-tracking-script.php?d=tracking-code&amp;mode=a-mode"></script>
<script type="text/javascript" src="/a-different-tracking-script.php?referrer-id=another-mode"></script>

Here is Googlebot's behavior on its last two visits. It's ignores new content pages altogether, looks for the tracking .php files and leaves.

GET / HTTP/1.0" 200 10780 "-" "Googlebot/2.1

GET /a-tracking-script.php?d=tracking-code&amp;mode=a-mode HTTP/1.0" 302 0 "-" "Googlebot/2.1

GET /a-different-tracking-script.php?referrer-id=another-mode HTTP/1.0" 200 563 "-" "Googlebot/2.1

GET /a-tracking-script.php?d=tracking-code&amp;mode=a-mode HTTP/1.0" 302 0 "-" "Googlebot/2.1

Will disallowing these files in robots.txt be sufficient to encourage Googlebot to ignore these scripts?

I've considered using the javascript above in an external script in a disallowed directory, maybe with document.write, but it seems like an awful kludge. Still I don't want Googlebot wasting time and ignoring real content....

What would be the best method to blind it to the presence of the scripts?



WebmasterWorld Senior Member 10+ Year Member

Msg#: 273 posted 12:51 am on Feb 9, 2004 (gmt 0)

robots.txt should help from my experience.

Global Options:
 top home search open messages active posts  

Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved