Forum Moderators: phranque

Message Too Old, No Replies

CGI bin, links, and spiders?

Do spiders pick up and follow your links when your script, etc is in the cg

         

ann

4:17 pm on Jul 6, 2002 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



Had a thought,

the last reincarnation of my link script, after the zeus problems from Google, is in the cgi bin.
The data logs cat pages add pages and main pages, both html and cgi are there.

My question is: Can the spiders crawl these html pages and follow the links as they are or do I HAVE to undergo another change. Hours and hours of hand coping in the urls....aarghhhhhh! :(

PLEASE tell me if they are all right, I hope, I hope, I hope!

Ann

Air

4:49 pm on Jul 6, 2002 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



If you are saying that you have pages that have URL(s) like [example.com...] then yes the spiders will see those just fine.

ann

5:08 pm on Jul 6, 2002 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



THANK YOU, AIR!!!!

I am your humble servant.....:)

Ann

OOPs!

The url is ..cgi-bin/link/somthing.cgi
Which then calls the html pages from the cgi bin where they have to be with this script where the links are added and stored then displayed via cgi call through the main.html page (which is also in the cgi bin)....Will that still work for the SE's?

JEEZE, I hope this is clear!
Ann

martinibuster

5:20 pm on Jul 6, 2002 (gmt 0)

WebmasterWorld Administrator 10+ Year Member Top Contributors Of The Month



I have a form that when filled out, takes the user to another page that says "Thank You" (via a perl script within the cgi bin).

If your page in question is linked in the same fashion as my above described Thank You page, then no, a spider won't find it.

ann

6:00 pm on Jul 6, 2002 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



No,

Everything, even thank you is in the cgi bin

Ann

Air

9:24 pm on Jul 6, 2002 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



>The url is ..cgi-bin/link/somthing.cgi

Ah then if all the URL(s) are of that form i.e.
cgi-bin/link/somthing.cgi
cgi-bin/link/page.cgi
cgi-bin/link/another.cgi
cgi-bin/link/sample.cgi
cgi-bin/link/this.cgi
cgi-bin/link/that.cgi

they will be seen fine by spiders.

martinibuster,
in your case I could see why the spider would not see the "thank you" page. Since it requires that the form be filled in before the "thank you" page is displayed, and spiders will not fill in and/or submit forms.

ann

9:43 pm on Jul 6, 2002 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



Thank you!

I will use this "free time" wisely!

I'm off to BINGO!

Ann