Forum Moderators: coopster & phranque

Message Too Old, No Replies

Robot Link Following

         

DavidT

7:25 pm on Feb 22, 2004 (gmt 0)

10+ Year Member



The axs logging script can log visits to external urls if the url is given as htt*://www.example.com/cgi-bin/axs/ax.cgi?http://www.externalsite.com/.

Googlebot can follow this link but I am using a directory script whose search engine strips the 'http://' part away from external urls to save resources, so that the above link won't work, it becomes htt*://www.example.com/cgi-bin/axs/ax.cgi?www.externalsite.com/.

I modified the axs script so that it would auto-add 'http://' to the tracked urls.

From this:

elsif ($mode eq 'redir') {
print "HTTP/1.0 302 Moved\015\012" if ($IIS);
print "Location: $nexturl\015\012\015\012";

To this:

elsif ($mode eq 'redir') {
print "HTTP/1.0 302 Moved\015\012" if ($IIS);
print "Location: [$nexturl\015\012\015\012";...]

Would search engines like Google still be able to follow and index the link if the 'http://' part is only included within the function of the script?

Damian

8:35 pm on Feb 22, 2004 (gmt 0)

10+ Year Member



It doesn't matter how you write the url behind the question mark. If any major search engine follows the link it will be because it follows the 302 redirect.