Welcome to WebmasterWorld Guest from 54.167.85.221

Message Too Old, No Replies

Why googlebot keep visiting banned sites?

I couldn't find an answer myself maybe you can

     

moftary

7:57 am on Aug 14, 2005 (gmt 0)

10+ Year Member



I have a few sites that got nuked from google recently.

Tracking the sites logs shows repetetive visits from some or one of googlebots, real googlebots not just faked user-agent ones. Of course when these sites were unbanned they were getting much more visits from all or most googlebots. Questions is, since these sites were banned, why do googlebots still visiting them?

goodroi

8:00 pm on Aug 15, 2005 (gmt 0)

WebmasterWorld Administrator goodroi is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



One reason could be they are trying to figure out if you are smart enough to know what upset them and if you have corrected the problem. Remember not all penalties are the same.

steveb

8:04 pm on Aug 15, 2005 (gmt 0)

WebmasterWorld Senior Member steveb is a WebmasterWorld Top Contributor of All Time 10+ Year Member



"why do googlebots still visiting them"

To see if they still suck (in googlebot's opinion)?

anttiv

7:24 pm on Aug 17, 2005 (gmt 0)

10+ Year Member



Googlebot visits regularly my banned domain and tries to open pages I have removed 6 months ago. Googlebot is like a dumb machine and it is not able to act like a human and review sites to see if some tricks have been removed. (Unless it is programmed to do so)

JuniorOptimizer

8:23 pm on Aug 17, 2005 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



Why exactly do they say they've forwarded your problem to the engineers if they know your site is banned and why it's banned?

moftary

10:40 pm on Aug 21, 2005 (gmt 0)

10+ Year Member



Googlebot is like a dumb machine and it is not able to act like a human and review sites to see if some tricks have been removed.

Actually it's more than a dumb machine.

It has been ten days so far since I modified my robots.txt to "force" googlebot not to spider what I "thought" google might consider as a duplicated content, but googlebots insists on spidering the disallowed contents.

My logs show that every particular bot downloads robots.txt then ironically spiders the pages, as if I asked him to spider them not the opposite!

To see if they still suck (in googlebot's opinion)?

Correct a mondo, and explains why does it insist on spidering pages that I told it not to spider.

Why exactly do they say they've forwarded your problem to the engineers if they know your site is banned and why it's banned?

I dont seem to understand the question.

Johan007

9:01 am on Aug 22, 2005 (gmt 0)

10+ Year Member Top Contributors Of The Month



I donít think the bot would have time to check to see if its banned or not, spiders have to be quick and cant afford to get bogged down. Even if I am wrong its got something to do with a technical issue.

Small Website Guy

1:36 pm on Aug 22, 2005 (gmt 0)

10+ Year Member



Obviously, the algorithm that determines what gets spidered is different from the algorith that determines what gets displayed in the SERPs.

Furthemore, many of the bans are based on algorithms which Google is always tweaking, so Google doesn't really want to totally lose the banned pages, it just doesn't want to display them at the moment.

asamuel

8:52 pm on Aug 23, 2005 (gmt 0)

10+ Year Member



I have been nuked too. Since 18th July my site has been out of the google serps. Strange things I have noticed:

I have many pages cached in google including the index page.
Googlebot is coming to the site every day (did 980 hits yesterday)

I had problem with code which has cleaned up mistaken dup content and emailed G, no reply so far..

So in anyones opinion am I banned or is it a question of waiting to get reindexed?

I am so confused, many people have different theories, just dont know who to believe!

Any ideas would be appreciated!

JuniorOptimizer

11:03 am on Aug 24, 2005 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



I finally got an email back from Google yesterday that strongly suggested I check out the webmaster guidelines. I did, made changes to the only things I can figure were wrong, and replied to the email. I told them what I changed and how I interpreted their guidelines. I also kissed their hind quarters a bit. Within an hour I had a reply that said once again, everything was being forwarded to the engineers for review. The whole thing has the feeling of talking to a human ( a cut and paster) but it does seem they have people reviewing the sites.
 

Featured Threads

Hot Threads This Week

Hot Threads This Month