homepage Welcome to WebmasterWorld Guest from 23.20.149.27
register, free tools, login, search, subscribe, help, library, announcements, recent posts, open posts,
Subscribe to WebmasterWorld
Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt
Forum Library, Charter, Moderators: goodroi

Sitemaps, Meta Data, and robots.txt Forum

    
Googlebot chasing .php tracking script
is robots.txt enough?
Edouard_H




msg:1527885
 7:05 pm on Feb 8, 2004 (gmt 0)

I recently added an ROI tracking program to a site and I'm pretty happy with the reports it produces. The following code is placed on the pages to implement the tracking (altered for this post):

<script type="text/javascript" src="/a-tracking-script.php?d=tracking-code&amp;mode=a-mode"></script>
<script type="text/javascript" src="/a-different-tracking-script.php?referrer-id=another-mode"></script>

Here is Googlebot's behavior on its last two visits. It's ignores new content pages altogether, looks for the tracking .php files and leaves.

GET / HTTP/1.0" 200 10780 "-" "Googlebot/2.1

GET /a-tracking-script.php?d=tracking-code&amp;mode=a-mode HTTP/1.0" 302 0 "-" "Googlebot/2.1

GET /a-different-tracking-script.php?referrer-id=another-mode HTTP/1.0" 200 563 "-" "Googlebot/2.1

GET /a-tracking-script.php?d=tracking-code&amp;mode=a-mode HTTP/1.0" 302 0 "-" "Googlebot/2.1

Will disallowing these files in robots.txt be sufficient to encourage Googlebot to ignore these scripts?

I've considered using the javascript above in an external script in a disallowed directory, maybe with document.write, but it seems like an awful kludge. Still I don't want Googlebot wasting time and ignoring real content....

What would be the best method to blind it to the presence of the scripts?

 

ThomasB




msg:1527886
 12:51 am on Feb 9, 2004 (gmt 0)

robots.txt should help from my experience.

Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About
© Webmaster World 1996-2014 all rights reserved