homepage Welcome to WebmasterWorld Guest from 54.204.141.129
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member
Home / Forums Index / Google / Google News Archive
Forum Library, Charter, Moderator: open

Google News Archive Forum

    
Does Google Spider cgi? or cgi-bin?
guillermo5000




msg:179812
 10:19 pm on Sep 21, 2003 (gmt 0)

I have a forum which I am trying to get spidered. I have removed the session from the URL and submitted a URL like this:

[mysite.org...]

Will Google visit this URL? Thanks.

 

hotmetal




msg:179813
 3:36 pm on Sep 22, 2003 (gmt 0)

It may or may not. They spider dynamic pages, but I don't know about semicolons. I had a site I switched over to use semicolons, but Google didn't grab any of the dynamic pages. I switched it back to ampersands, and the next day Google tried to grab some semicolon pages, whatever that means.

Gus_R




msg:179814
 3:38 pm on Sep 22, 2003 (gmt 0)

Check logs for googlebot at that page, if don't, consider something like mod_rewrite

GoogleGuy




msg:179815
 4:34 pm on Sep 22, 2003 (gmt 0)

That looks pretty good to me, guillermo5000, but I'll try to ask around as well. Yeah, judging from the results for
inurl:ikonboard inurl:act
it's at least possible--I see lots of similar url patterns indexed.

If you want other search engines to crawl you, you might thing about what another poster suggested too--having static-looking urls might help for other engines. But I think you'll be fine for Google.

Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Google / Google News Archive
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved