Welcome to WebmasterWorld Guest from 54.146.248.111

Forum Moderators: open

Message Too Old, No Replies

Googlebot is Crawling

Starting seeing some action

     

fiestagirl

5:17 am on Jul 31, 2002 (gmt 0)

10+ Year Member



Seeing a few requests for the home pages of a few sites! Starting a couple of hours ago.

cYbErDaRk

9:08 am on Jul 31, 2002 (gmt 0)

10+ Year Member



Yes!

Though it seems I've banned (or 20k droped) I've seen in my log yesterday it came to my website and got 365 pages! That's good, 'cause many of the URL are not on google index today, so it's seems some of us dropped on last update are going to be "taken" again.

Or not? :)

Regards (and sorry for my English)

rfgdxm1

9:21 am on Jul 31, 2002 (gmt 0)

WebmasterWorld Senior Member rfgdxm1 is a WebmasterWorld Top Contributor of All Time 10+ Year Member



Google dropped my site in the garbage like a box of used Kleenex last update, even though I was crawled. My site was just again crawled by Googlebot today. However, I'm not getting my hopes up. :(

TonyM

9:31 am on Jul 31, 2002 (gmt 0)

5+ Year Member



I seem to have had a similar experience
My site was orignally PR5-6 went to PRO on update which I couldn't understand, later went grey,now I note Gogglebot visited 550 pages in last days.

Although none of my pages are in index yet I hope I may get back in sometime and I wasn't banned.

One hopeful Google fan

KakenBetaal

10:09 am on Jul 31, 2002 (gmt 0)

10+ Year Member



Please tell us your home page PR and high/low estimate if you see crawling. That data is of interest to a few of us out here.

brotherhood of LAN

10:12 am on Jul 31, 2002 (gmt 0)

WebmasterWorld Administrator brotherhood_of_lan is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



nice timing, ive just been uploading those fancy absolute divs and light CSS, hopefully google will find the newer slim code much more tasty food to eat :)

For once, google has crawled at a convenient time.

cYbErDaRk

10:23 am on Jul 31, 2002 (gmt 0)

10+ Year Member



KakenBetaal:

PR 6. If someone is interested, can see googlebot evolution on my website (see my profile) /logs/googlebot_log

If I see some change on www2 or www3, I'll tell you

taxpod

10:28 am on Jul 31, 2002 (gmt 0)

10+ Year Member



PR7 being crawled as of late last night.

To those who feel hopeless cause they were dropped last update, don't despair. It's very important for my google referrals to have a lot of pages in the index. Two or so months ago, I got cut by some 70% of pages. Then things turned around and the latest update has made a huge difference.

When you get cut from the index or have a lot of pages get dropped, assuming you aren't doing anything nasty, take a good look at your code and your internal navigation. Get a few more inbound links from anywhere you can and maybe in a month or two you might be singin a happy tune.

Go Googlebot, slurp lots and lots of pages!

KakenBetaal

11:21 am on Jul 31, 2002 (gmt 0)

10+ Year Member



Thanks Cyberdark, that was interesting. Nice looking site you have there! It looks like my crawl will be 12-24 hours behind yours as I've just had a few more pages taken. Hey, hey, hey!

I'm hoping the bot will see my newly installed forum and go take some of those pages as well.

On nav structure, I have both a site map (aimed at Googlebot more than users), and a nav bar link to all my main tier 2 pages from every page on the site. That seems to work fairly well for me, and nearly all my pages are indexed regularly. Thanks to WebmasterWorld and all you helpful users for the site map tip - it helped to get far more of my pages indexed.

What may also help my site to be indexed is that I have quite a few external deep links.

Rugles

3:19 pm on Jul 31, 2002 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



Googlebot grabbed a 1000 pages last night.

EliteWeb

6:59 pm on Jul 31, 2002 (gmt 0)

WebmasterWorld Senior Member eliteweb is a WebmasterWorld Top Contributor of All Time 10+ Year Member



PR7 and crawling nonstop many sites.
One has PR 4
and other that was hit is PR 5

I dont have Low PR sites really so those are some results

Gibble

7:17 pm on Jul 31, 2002 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



All it's done is grab my robots.txt files...I'm waiting for the crawl now.

And I'm impatient!

stuntdubl

7:55 pm on Jul 31, 2002 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



Does anyone have any suggestions on the best freeware tracking programs? I'm currently using extreme tracking, but I'm sure there has to be something better. I want to be able to watch for the googlebot too!

kawikadave

8:29 pm on Jul 31, 2002 (gmt 0)

10+ Year Member



PR5 site - Googled crawled the whole thing early this morning - about 75 pages.

Hemsell

6:16 pm on Aug 4, 2002 (gmt 0)

10+ Year Member



pr0 site. Many new incoming links.
Googlebot has been crawling all over me since 2002-07-31 05:08:37.000
and last entry on 2002-08-04 03:35:25.000
I have a dynamic page the SQL Server generates here www.hemsell.com/resources/googlebot.htm if anyone is interested

cYbErDaRk

9:57 pm on Aug 4, 2002 (gmt 0)

10+ Year Member



Update: up to 20000 pages crawled. It hung my SQL server (more than 500 petitions/hour) :(

Let's hope, on August update, site reapears there :)

Big_Balou

10:26 pm on Aug 4, 2002 (gmt 0)

10+ Year Member



Googlebot grabbed robots.txt on friday morning and looks like it finished spidering the entire site last night....with the help fo a new site map....thanks WW.

Marcia

10:39 pm on Aug 4, 2002 (gmt 0)

WebmasterWorld Senior Member marcia is a WebmasterWorld Top Contributor of All Time 10+ Year Member



stuntdubl, dig through the Tracking forum [webmasterworld.com], you'll find a lot of them recommended.

tesla

11:41 pm on Aug 4, 2002 (gmt 0)

10+ Year Member




PR5, and it has been crawling for the last several days

mortalfrog

3:59 am on Aug 5, 2002 (gmt 0)

10+ Year Member



if it actually hung your SQL server, you might want to look into using a template engine that is capable of caching repetitive SQL queries. The Smarty template engine is extremely easy to use if you have PHP on your system, and does a whole lot more than caching - but if google is halting your server, you should get some kind of query caching going ASAP to avoid losing traffic etc. or burning out your CPU ;D
 

Featured Threads

Hot Threads This Week

Hot Threads This Month