Forum Moderators: open

Message Too Old, No Replies

Google Spidering CGI and SID's

Did I miss something?

         

stuntdubl

5:49 pm on May 6, 2004 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



I guess maybe I haven't paid close enough attention, but I have just noticed recently that Google is spidering and indexing cgi shopping stores and even session id's.

A few recent sites that I have used the "site:" command on have shown nearly all of their cgi generated shopping cart pages.

I know G has been working hard on their spiders, and maybe I should just read forum 3 more often, but has this been discussed in depth?

Most of the pages currently have no PR, so I'm guessing this is something fairly new or they are just down in the heirarchy level far enough where they are < PR1.

I searched WW a bit, and didn't find much discussing this. I had seen discussions on G spidering Javascript now, but wondering if anyone could point me to discussions on CGI and SID spidering by G.

stuntdubl

7:01 pm on May 6, 2004 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



Okay, I found some old posts on CGI spidering (looks like it's been around much longer than I had thought), but I didn't see much on Session Id's. I was under the impression that G didn't mess with them, and this was new to me. Guess I was a bit behind the times on this one, and I probably would still view cgi pages and such as a negative (I'm guessing they still don't pass PR with their links).

Anyhow, if anyone has flagged any good posts on the subject I would appreciate it.

ILLstyle

8:51 pm on May 7, 2004 (gmt 0)

10+ Year Member



stuntdubl

One of my sites uses CGI for all its shopcart pages.
Some of the pages are over 6 years old. early last month I noticed these pages being spidered and indexed for the first time. They now show up in the serps and in froogle. it's funny but your post was the first one I have seen that mentioned this. I was kind of surprised no one else posted on this new (at least to my web site) spidering.

doc_z

9:35 am on May 8, 2004 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



Spidering CGI pages isn't new: Does Google spider .cgi pages? [webmasterworld.com]

Most of the pages currently have no PR....

There're problems with the PR shown for dynamic pages.

ILLstyle

3:36 am on May 9, 2004 (gmt 0)

10+ Year Member



doc_z
why the link to a post form 2002, old news bro. I don't care what GG say, GB no spider my CGI untill last month.