Welcome to WebmasterWorld Guest from 107.23.37.199

Forum Moderators: coopster & jatar k & phranque

Message Too Old, No Replies

Robot Link Following

     
7:25 pm on Feb 22, 2004 (gmt 0)

Full Member

10+ Year Member

joined:Dec 13, 2002
posts:275
votes: 0


The axs logging script can log visits to external urls if the url is given as htt*://www.example.com/cgi-bin/axs/ax.cgi?http://www.externalsite.com/.

Googlebot can follow this link but I am using a directory script whose search engine strips the 'http://' part away from external urls to save resources, so that the above link won't work, it becomes htt*://www.example.com/cgi-bin/axs/ax.cgi?www.externalsite.com/.

I modified the axs script so that it would auto-add 'http://' to the tracked urls.

From this:

elsif ($mode eq 'redir') {
print "HTTP/1.0 302 Moved\015\012" if ($IIS);
print "Location: $nexturl\015\012\015\012";

To this:

elsif ($mode eq 'redir') {
print "HTTP/1.0 302 Moved\015\012" if ($IIS);
print "Location: [$nexturl\015\012\015\012";...]

Would search engines like Google still be able to follow and index the link if the 'http://' part is only included within the function of the script?

8:35 pm on Feb 22, 2004 (gmt 0)

Preferred Member

10+ Year Member

joined:Dec 19, 2000
posts:366
votes: 0


It doesn't matter how you write the url behind the question mark. If any major search engine follows the link it will be because it follows the 302 redirect.