homepage Welcome to WebmasterWorld Guest from 54.237.78.165
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member
Home / Forums Index / Google / Google News Archive
Forum Library, Charter, Moderator: open

Google News Archive Forum

This 42 message thread spans 2 pages: < < 42 ( 1 [2]     
Weird bot behavior
ukgimp

WebmasterWorld Senior Member 10+ Year Member



 
Msg#: 15561 posted 3:49 pm on Jul 16, 2003 (gmt 0)

Article [securitynewsportal.com]

Requests for:
[error] [client 64.68.82.31] File does not exist: /public_html/maillist/maillist_signin.asp

If this is Google, have they turned their technology loose on the web to create SPAM mailing list to sell in a quest for even greater profits?

paranoid or what, but this is what is out there.

 

ogletree

WebmasterWorld Senior Member ogletree us a WebmasterWorld Top Contributor of All Time 10+ Year Member



 
Msg#: 15561 posted 4:35 pm on Jul 17, 2003 (gmt 0)

We have a DHTML menu that google does not spider at all. We had to create a sitemap to get spidered. The problem is we had to modify for Google. If a search engine could provide good results from all sites they would beat out Google. There should be no such thing as SEO if Google was doing their job. One of the biggest online computer etailers does not have anything in Google but weird URLS that mean nothing (no title no description) for all their products. The big guys that use TV and other advertising just don't care about Google. Big companies just buy their ads. I am sure some people are glad this happens but it means that Google is missing a lot of good stuff. Search engines are or soon will be nothing but advertisments. The web is not what it used to be it is a huge consumer juggernaught.

Chicago

WebmasterWorld Senior Member 10+ Year Member



 
Msg#: 15561 posted 12:59 am on Jul 18, 2003 (gmt 0)

and maintains his position.

i know for a fact

Falling over in my seat at what i just read. I have never seen a more meaningless debate have such significance on WW.

AhmedF

10+ Year Member



 
Msg#: 15561 posted 6:40 am on Jul 19, 2003 (gmt 0)

why does everyone have to call it a conspiracy theory? "conspiring" to do things that cause business to grow is also called 'good business practices' by others.

I used to say it before, but now I don't - the phrase is over-abused.

Dolemite

10+ Year Member



 
Msg#: 15561 posted 3:41 pm on Jul 20, 2003 (gmt 0)

Well, there is a 1-to-1 correlation some how between visiting unindexed new pages with the toolbar and GoogleBot showing up. It has happened to too many people to be an accident. Some how they are linked - either directly from gbot putting the url in the spiders inbox, or via referral leaking - but I know for a fact, they are linked.

I didn't believe the whole toolbar indexing thing...GoogleGuy's reassurances to the contrary were convincing, until now. I've had a site in development this past week...no incoming links. 4 pages of a subdomain (definitely no incoming links) were crawled, though none are indexed yet. I don't buy the referral log thing, since these pages didn't have any outgoing links at the time, either.

dmorison

WebmasterWorld Senior Member 10+ Year Member



 
Msg#: 15561 posted 3:45 pm on Jul 20, 2003 (gmt 0)

I don't see why people make such a big thing of it. It seems an obvious thing for Google to do.

Google aim to index as much of the web as possible; and would therefore want to use any source of URLs available to them.

If Toolbar reports a URL not previously known to Google they would be mad not to crawl it.

Dolemite

10+ Year Member



 
Msg#: 15561 posted 3:57 pm on Jul 20, 2003 (gmt 0)

I don't see why people make such a big thing of it. It seems an obvious thing for Google to do.

Why would they deny it then? I agree, its not that big of an issue, I'd just like to know what the reality is, and my experience is leading me to question Google's version of reality.

There are some times when I don't want Google looking at what I'm looking at, though...site development being a good example.

dmorison

WebmasterWorld Senior Member 10+ Year Member



 
Msg#: 15561 posted 4:04 pm on Jul 20, 2003 (gmt 0)

Why would they deny it then?

That's what I don't understand!

Although to be fair I don't think Google have denied it as such, more GG stated that he didn't _think_ it happened; and also went on to state that nothing in their privacy policy would prevent them from doing so.

If you ask me it's not the sort of thing that has anything to do with a Privacy Policy anyway. A URL is a URL; if the web service handling Toolbar queries simply chucks each URL into Googlebot's Inbox with no reference to the Toolbar user that introduced it then there is no reason for it to even come up within a privacy statement.

Clark

WebmasterWorld Senior Member 10+ Year Member



 
Msg#: 15561 posted 11:57 pm on Jul 20, 2003 (gmt 0)

They would be even more crazy not to index pages that the MediaPartners Googlebot comes after because those pages are definitely being read and seen by humans. Why would it not use that googlebot data once it's collected? I know they say it isn't, and I can't prove they do, but I can't imagine the two bots wouldn't share that data.

ThatAdamGuy

10+ Year Member



 
Msg#: 15561 posted 6:46 pm on Jul 29, 2003 (gmt 0)

Okay, here's my experience.

A few weeks ago, I created a new forum (message board) on my site.

I do have the Google Toolbar (2.0 beta) on my system, and I also have AdSense ads on every page of my forum.

Here is what I observed repeatedly:
Within two minutes of creating a new discussion thread, a GoogleBot showed up to that exact (and previously non-existent) URL.

This, then, would normally indicate one of the following:
1) The URL was 'leaked' from the toolbar.
2) AdSense was 'inviting' the bot there.
3) Both of the above.

Since GG has unequivocally denied that #1 is the case, it must mean that he's either mistaken (not likely, IMHO) or lying (even less likely, IMHO) or that the answer is #2 (the bot visit is AdSense related).

Unfortunately, I'm currently too lazy to uninstall the toolbar from my browser to test this, but for those who are concerned about the mysterious appearance of Googlebots... perhaps you're also running AdSense ads on those pages?

Just something to think about.

WebWalla

WebmasterWorld Senior Member 10+ Year Member



 
Msg#: 15561 posted 6:56 pm on Jul 29, 2003 (gmt 0)

Visiting a page with AdSense code on it may cause the Mediapartners bot to visit, but not the normal index bots, IMO.

mayor

WebmasterWorld Senior Member 10+ Year Member



 
Msg#: 15561 posted 7:32 pm on Jul 29, 2003 (gmt 0)

GG >> Referral leaking is a common way that we find leaks to unindexed pages

So if I have the toolbar active my browser, go to my new website, then use the search window on the toolbar while I'm there, does that create a referral from the new site?

farside847

10+ Year Member



 
Msg#: 15561 posted 8:03 pm on Jul 29, 2003 (gmt 0)

Holly GoogleBot Batman!

(since my other attempts of making a new thread were
refused, grumble, censorship, grumble, I will post here.)

65k pages downloaded since 1AM today, an average of 104 pages per minute, 1.7 pages per second. This onslaught is slowing my server from a normally zippy rabbit, to a slow laboring turtle.

My only hope is that this might mean an update to my 3 month old PR and backlink values.

Anyone else seeing this kind of traffic?

This 42 message thread spans 2 pages: < < 42 ( 1 [2]
Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Google / Google News Archive
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved