homepage Welcome to WebmasterWorld Guest from 54.166.122.86
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Pubcon Platinum Sponsor 2014
Home / Forums Index / Marketing and Biz Dev / Deprecated - Search Engine Submission
Forum Library, Charter, Moderator: open

Deprecated - Search Engine Submission Forum

    
How do search engines deal with cookies and/or GET?
dahamsta

10+ Year Member



 
Msg#: 7 posted 12:59 am on Sep 3, 2000 (gmt 0)

Generally speaking, when a search engine comes to a site, does it accept or ignore cookies?

In the case of a site I'm working on at the moment, if the engine doesn't accept cookies, it'll switch to passing a session id via GET, so how will the engine deal with that? I know that engines used to ignore URL's with QUERY_STRING's, because they used to get sucked into a loop with the end result of bringing both servers down, but what about now?

Do they ignore pages with QUERY_STRING's altogether? Do they follow the links anyway, but use some kind of intelligence to ignore loops? Or do they ignore the QUERY_STRING, and index the pages anyway? If it's the latter, they'll end up creating a load of useless session files on my machine, and I want to avoid that.

Any help appreciated. Ta.

adam

 

Brett_Tabke

WebmasterWorld Administrator brett_tabke us a WebmasterWorld Top Contributor of All Time 10+ Year Member



 
Msg#: 7 posted 2:30 am on Sep 3, 2000 (gmt 0)

Hi Adam, welcome to the board. As far as I know, there are no major se spiders that use cookies in anyway.

Yes, query strings are still ignored by all the majors.

herb

10+ Year Member



 
Msg#: 7 posted 9:49 pm on Sep 4, 2000 (gmt 0)

Brett, am I misunderstanding your post when you say that:

"there are no major se spiders that use cookies in anyway."

or am I taking it out of context?

Excite, Google, Infoseek, Webcrawler and Hotbot are all setting cookies from the submit page. What am I missing?

Air

WebmasterWorld Senior Member 10+ Year Member



 
Msg#: 7 posted 10:14 pm on Sep 4, 2000 (gmt 0)

Hello Herb, welcome to the board.

Brett is talking about spiders accepting cookies when crawling a site. Getting them set on your machine when you visit a search engine page is a different story.

But if you push Brett, I'm sure he'll tell you that the cookies set on your machine by the engines do nothing also, except maybe tell them one of the elements of a real browser is present. I have turned off cookies many times at submission without ill effects.

2_much

WebmasterWorld Senior Member 10+ Year Member



 
Msg#: 7 posted 12:40 am on Feb 28, 2001 (gmt 0)

Is this still the case?

So if you submit many index pages and clean cookies between submissions, the engines won't penalize you for over submitting and the submission goes through okay?

2_much

WebmasterWorld Senior Member 10+ Year Member



 
Msg#: 7 posted 12:49 am on Feb 28, 2001 (gmt 0)

Yes 2Much, you are right on, this is the correct :)

FreeBee

10+ Year Member



 
Msg#: 7 posted 2:13 am on Feb 28, 2001 (gmt 0)

<if you submit many index pages and clean cookies between submissions, the engines won't penalize you for over submitting and the submission goes through okay?>

Surely not? Would cookies have anything to do with this?

If you oversubmit in a given period from completely different machines on independent IP's you'll still get hammered for oversubmission, cookies or no cookies.

FreeBee

10+ Year Member



 
Msg#: 7 posted 2:20 am on Feb 28, 2001 (gmt 0)

I should have qualified last post - oversubmission is a problem where submit limits apply of course (AV, Excite, Hotbot....)

Brett_Tabke

WebmasterWorld Administrator brett_tabke us a WebmasterWorld Top Contributor of All Time 10+ Year Member



 
Msg#: 7 posted 7:35 am on Feb 28, 2001 (gmt 0)

2_much, there is a bit of debate about it right now. I think it is wise to accept the cookie. The cookie itself isn't much of burden on resubmission, your ip is the key. It is easier for them to tag a submission with your IP address than it is a cookie. The only two engines I worry about in that regard, is Alta, and Excite. I've been convinced Alta is tracking IP's and tossing submissions after #xyz per day from the same IP (I say it is 50, but don't quote me because I submit far greater than that some days).

Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Marketing and Biz Dev / Deprecated - Search Engine Submission
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved