homepage Welcome to WebmasterWorld Guest from
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member

Home / Forums Index / Google / Google News Archive
Forum Library, Charter, Moderator: open

Google News Archive Forum

This 49 message thread spans 2 pages: 49 ( [1] 2 > >     
GoogleBot didn't visit me after 23rd
Anyone same here?

 8:29 am on Dec 27, 2002 (gmt 0)

I have a new site and GoogleBot seems to visiting my site almost everyday since the start of december. But suddently it stop on 23rd and never come back. No even christmas. Anyone has the same problem.

I recall I add a robot.txt at that period that ask all robots to exclude a specific file which do meta refreshes to sites located outside. I just don't think this robot.txt is the main reason that stop google from coming back. Anywhere, I have remove it.

Just want to see if anyone facing the same problem here.



 8:54 am on Dec 27, 2002 (gmt 0)

I had such a situation 2 month ago with few of my domains, and after dance i found those domains banned.


 9:19 am on Dec 27, 2002 (gmt 0)

On 15rd of dicember I got a freshtag on my site and GoogleBot visited it every two days but at 21rd stop and now the index show the old cache


 10:55 am on Dec 27, 2002 (gmt 0)

That's strange...before the 23rd Googlebot would drop by very early in the morning, 6-7 AM but since then he's crawling at 11-12 in the morning. Maybe he's taking a holiday?


 12:26 pm on Dec 27, 2002 (gmt 0)

If googlebot taking holiday, then it is a long one as it never return until today!

troels nybo nielsen

 12:56 pm on Dec 27, 2002 (gmt 0)

I am one of those lucky webmasters that are visited daily by freshbot. Without being quite sure my guess about the reason would be that so many of my incoming links are from pages that are updated daily. So even if for a time I myself do not make any updates I am still visited, only freshbot stops digging deep into the websites.

What kind of incoming links do you have, jamesyap?


 1:13 pm on Dec 27, 2002 (gmt 0)

It is a new site that is not index yet, I do have around 10 links from other sites, include DMOZ.


 2:56 pm on Dec 27, 2002 (gmt 0)

I think the freshbot reduced his work after the 23rd. The crawler-activity on the site controlled by me is only by 30% of the normal access.


 3:13 pm on Dec 27, 2002 (gmt 0)

I noticed a similar activity...FreshBot normally gets my sites every day..lots of incoming high PR sites, and like clockwork. The last week has been every other day for some reason...server definitely online, other bots requesting the heck out of my robots file, so Im at a loss. Been listed for a few years and only everflux effects with new content and links out. Just struck me as odd behaviour.


 3:18 pm on Dec 27, 2002 (gmt 0)

For what its worth, freshbot hit me on the 26th, and it is showing in the results.


 5:23 pm on Dec 27, 2002 (gmt 0)

I am still waiting for googlebot to come after 23rd, but no news. :(


 5:49 pm on Dec 27, 2002 (gmt 0)

This is completely normal for newer sites. The important thing is to not freak out if the bot doesn't come every day, or the Googledance is late, or your PR drops one point, or you don't immediately make the top ten, or your rankings sometimes jump around a little bit.

Patience in all things.


 7:09 pm on Dec 27, 2002 (gmt 0)

The time to wait is killing me, I have been waiting for more than a month! Nothing can be done.


 7:45 pm on Dec 27, 2002 (gmt 0)

I had also strange behaivor:

1) site was visited first time at middle of November and every few days it only took robots.txt and / a few times google took the whole site (220 pages).
3) We upgraded inital site to production site (around 14K pages)
4) At begining of December, google crawled night and day, but only indexed 220 pages (for the first time) while downloading almost 3.5K pages from avaliable 14K. (near 200Mb of traffic for 2 days)
5) A day after all 220 pages disapears from google index, currenlty only root page is indexed.
6) Ever since the big crawl Google visits the site every day taking only (robots.txt and /).

Can someone please explain this?


 9:28 pm on Dec 27, 2002 (gmt 0)

Anything crawled in December, or even the latter part of November, will not be in Google's main database yet. The database update (the "Googledance") is typically around the end of the month (i.e. soon). Again, PATIENCE.


 9:31 pm on Dec 27, 2002 (gmt 0)

I don't think we'll see new google dance 2002.


 3:47 am on Dec 28, 2002 (gmt 0)

Have google skip any dance in a particular month in history? I don't think so.


 5:46 am on Dec 28, 2002 (gmt 0)

>> Have google skip any dance in a particular month in history? I don't think so<<

Yes they have, early this year or late last i think...


 6:02 am on Dec 28, 2002 (gmt 0)

Hopefully they won't skip this dance, I have wait for a month to see what a google dance (everflux) really is because I have never seen one.


 10:36 am on Dec 29, 2002 (gmt 0)

Are there any signs that can predict a google dance?


 3:58 pm on Dec 29, 2002 (gmt 0)

Barnyard animals often become agitated before a Googledance.


 4:15 pm on Dec 29, 2002 (gmt 0)

haha, then googledance must be more exciting then earthquake.


 4:31 pm on Dec 29, 2002 (gmt 0)

Ask Barney Google:

last one to the gossip fence is a rotten egg:


I think there's some hidden meaning in those ;-)


 4:34 pm on Dec 29, 2002 (gmt 0)

She has been hitting us as always - every few days... We have a PR of 6.


 5:00 pm on Dec 29, 2002 (gmt 0)


for my PR5-.de-domain
I have registrated

fully deep crawl between 2nd and 4th Dec
a second (only partly) deep crawl on 11th Dec

fresh bot visted me last
on 27th Dec - only 1 hit to the home-page
26th about 30% of pages

25th about 30%
24th about 100%
23rd about 30%
22nd about 50%


 6:13 pm on Dec 29, 2002 (gmt 0)

Looks like googlebot take a break after 27th.

But WHY googlebot skip my site from 24th to 17th.

Make me MAD MAD MAD~~~


 10:03 pm on Dec 29, 2002 (gmt 0)

>> I had such a situation 2 month ago with few of my domains, and after dance i found those domains banned <<

I don't think that's it. I haven't seen a Google bot in the last couple of days either, and I'm far too nice of a guy to be banned. :-)


 8:09 am on Dec 30, 2002 (gmt 0)

Patience is "definitely" a virtue.
Still I suspect the folks at Google have a quirk similar to that of a "sadistic comedian's".
With all their attention to the holidays, I wouldn't be a bit surprised if "the dance" was activated on New Years Eve.
So the "impatient" would focus their attentions on Google, and forsake the upcoming holiday.

My advice is to step away from your computer, put on a silly hat, then dance and laugh with a pretty girl dressed in one of those slinky sequined dresses.
And have fun on New Years Eve!
That's what I'll be doing.

You can watch the Google dance if you want, but it will be another week until the index "falls into place".
So what's the point?


 8:21 am on Dec 30, 2002 (gmt 0)

Hey peoples,

Google says that if I type <META NAME="googlebot" CONTENT="noindex,nofollow"> in my tags, Google will avoid scanning the site. But what-if I put <META NAME="googlebot" CONTENT="index,follow">
in my site....will this make it more likely that google will take it because I recognized it? mmmmmmm....just curious. :) Only googleguy would know. hehehe.

Ken Eynon


 11:26 am on Dec 30, 2002 (gmt 0)

Looks like all crawlers go to sleep as bears at winter.

1. InternetSeer Dec. 30, 2002 914 (4.8%) 898 (15.6%)
2. TurnitinBot Dec. 29, 2002 1,912 (10.1%) 20 (0.3%)
3. Mercator Dec. 29, 2002 2,202 (11.6%) 20 (0.3%)
4. Google Search Engine Dec. 27, 2002 12,626 (66.8%) 4,628 (80.2%)
5. FAST-WebCrawler Dec. 26, 2002 8 (0.0%) 8 (0.1%)
6. Baiduspider+(+http: Dec. 25, 2002 2 (0.0%) 2 (0.0%)
7. Larbin Dec. 25, 2002 4 (0.0%) 4 (0.1%)
8. BaiDuSpider Dec. 24, 2002 4 (0.0%) 2 (0.0%)
9. SurveyBot Dec. 15, 2002 6 (0.0%) 6 (0.1%)
10. WiseNut Search Engine Dec. 15, 2002 171 (0.9%) 108 (1.9%)
11. Internet Archive Dec. 10, 2002 80 (0.4%) 50 (0.9%)

This 49 message thread spans 2 pages: 49 ( [1] 2 > >
Global Options:
 top home search open messages active posts  

Home / Forums Index / Google / Google News Archive
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved