Welcome to WebmasterWorld Guest from 54.242.173.65

Forum Moderators: open

Message Too Old, No Replies

Is Freshbot now Deepbot?

The line is getting drawn ever thinner

     
4:18 pm on May 22, 2003 (gmt 0)

Senior Member

WebmasterWorld Senior Member trillianjedi is a WebmasterWorld Top Contributor of All Time 10+ Year Member

joined:Apr 15, 2003
posts:7242
votes: 0


I've seen several postings about this now in the last few days, although this is my first actual experience of it.

I'm being hit very hard by google's freshbot at the moment, and going deep too. At first glance at what is currently going on with the little guys, I had to check and double check that the IP's were 64.... (they are).

It's behaviour, in terms of hard hitting and depth of crawl (it's going through the entire site) is more like the character of the old deepbot.

In fact, it's identical behaviour to deepbot the last time it crawled this site back in April.

I'm interested in hearing from others who are seeing the same.

TJ

6:14 am on June 6, 2003 (gmt 0)

Junior Member

10+ Year Member

joined:Mar 19, 2003
posts:125
votes: 0


I added pages and pages of fresh content 3 days ago - it is all showing up in the SERPS now so freshbot is definitely crawling.
6:19 am on June 6, 2003 (gmt 0)

Moderator from GB 

WebmasterWorld Administrator mack is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:June 15, 2001
posts:7564
votes: 4


I have noticed a few new pages getting in, But they aren't being dated the way fresh listings used to be. Don't know if this is something for the future or just a temporery glitch.

Mack.

7:35 am on June 6, 2003 (gmt 0)

Junior Member

10+ Year Member

joined:May 15, 2003
posts:142
votes: 0


Freshie's in my logs too -- and still asking for pages I moved months ago. Obviously the lessons of April were not learned.

301
301
301

9:33 am on June 6, 2003 (gmt 0)

Moderator from AU 

WebmasterWorld Administrator anallawalla is a WebmasterWorld Top Contributor of All Time 10+ Year Member

joined:Mar 3, 2003
posts:3725
votes: 9


Had freshbot 7 times since 1 June on a PR6 site (has a PR7 deeper) and 4 times on another PR6 site. There was a gap of 1-2 weeks since the previous visit.

- Ash

4:03 pm on June 6, 2003 (gmt 0)

Junior Member

10+ Year Member

joined:Apr 11, 2003
posts:149
votes: 0


Freshie has been hitting one of my sites for a few months now. I put up some new pages a few weeks back but it seems he just doesn't want to take a look! On my other sites freshie is always eager for a new tipple and dives straight into the the new content pages. Anyone else notice this? The links are right there but he just doesn't want to know, instead he keeps sniffing out the old pages but there is nothing really going on there!

Any ideas?

4:13 pm on June 6, 2003 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:Feb 22, 2003
posts:1483
votes: 0


No problem getting new stuff here. Do you have links to these new pages on your strongest PR page?
4:54 pm on June 6, 2003 (gmt 0)

Junior Member

10+ Year Member

joined:Apr 11, 2003
posts:149
votes: 0


Yeah on my idex page PR5, same as other sites. Can't understand it but something is causing freshie to turn his nose up!
7:29 pm on June 9, 2003 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:Nov 8, 2002
posts:2335
votes: 0


Freshbot keeps hitting the same pages again and again. I know we won't know until GG decides to inform us or the event is upon us when a real crawl will come again, but I was wondering if any new sites are getting fresh pages crawled...or is G stale for new sites as well?
7:34 pm on June 9, 2003 (gmt 0)

Preferred Member

10+ Year Member

joined:Apr 18, 2003
posts:618
votes: 0


I created about a dozen new pages yesterday night and today morning and freshbot came about 5 minutes later and picked up all of them. ;)
7:41 pm on June 9, 2003 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:Nov 8, 2002
posts:2335
votes: 0


I had a 150 pages crawled in the last day or so but I think there's not a new one in the bunch. Oh well.
7:48 pm on June 9, 2003 (gmt 0)

Preferred Member

10+ Year Member

joined:Apr 18, 2003
posts:618
votes: 0


I think it is feeling guilty in my case since its elder sister (brother?) lost all except the index page it crawled during April. Trying to make up for that, perhaps.
9:05 pm on June 9, 2003 (gmt 0)

Junior Member

10+ Year Member

joined:May 15, 2003
posts:142
votes: 0


My site is about PR1 (the best in my cat are PR3, and several years older than mine, so I'm not worried) so freshbot doesn't come by to get my new pages (kinda sucks). But the freshie-as-deepbot has been slowing crawling my site this month, and it keeps asking for pages that were 301-redirected several months ago.

I know it's already been established that Googlebot doesn't seem to "know" about the April deepcrawl, yada yada. I'm just one more confirmation.

It is frustrating, though.

9:18 pm on June 9, 2003 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:Nov 8, 2002
posts:2335
votes: 0


Yeah. We may have to wait what, another 4 weeks or so? Plenty of time to work on more food for "G". I know I've got lots of food waiting :)
9:30 pm on June 9, 2003 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:June 26, 2000
posts:2176
votes: 0


>>i'm seeing freshbot daily

But are you seeing the pages the FB grabs show up in the index? From what I'm seeing, the FB system is only funtioning properly for those pages coming from sites that were already in the Feb/March db.

On new sites, FB is crawling regularly, but those new pages aren't showing up.

9:34 pm on June 9, 2003 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:May 24, 2003
posts:814
votes: 0


yes i'm getting hundreds of pages crawled nearly every day and only my homepage is being updated in the index.
9:36 pm on June 9, 2003 (gmt 0)

New User

10+ Year Member

joined:Feb 25, 2003
posts:27
votes: 0


I'm getting hits to PDF files from freshbot. Is this normal freshbot behaviour? I can't rememeber seeing this before.
9:41 pm on June 9, 2003 (gmt 0)

Preferred Member

10+ Year Member

joined:June 15, 2002
posts:396
votes: 0


freshbot is cwraling but, no fresh tags on my sites.(these sites are a few months old and got struck down with alot of other sites when googlegonewild made its grand apperents).

teeceo.

8:59 am on June 10, 2003 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:Feb 5, 2003
posts:689
votes: 0


I see freshie everyday. new pages crawled, not showing up in index.

Changes I make on the homepage are grabbed and in the index within 2 days.

On another site I have, created 2 months back, while fb/gb has visited it several times, still does not show up in the index. not even the homepage. while ATW has eaten up all pages.

9:57 am on June 10, 2003 (gmt 0)

Preferred Member

10+ Year Member

joined:Apr 1, 2003
posts:438
votes: 0


I seeing Freshie a lot too and some pages added.... it seems

This site first got PR4 in April, then PR evaporated during DOM. Presently keeps flipping between showing 153 pages allinurl and 98!

Have been getting traffic from new pages and not finding them listed amongst the 153...

Weird sh1t!

8:30 am on June 10, 2003 (gmt 0)

Junior Member

10+ Year Member

joined:Jan 5, 2003
posts:86
votes: 0



I have a brand new website. Googlebot has been spidering it for the last three weeks. I log everything. Currently googlebot has fetched about 60000 pages from my website. But only a few are visible in google. When doing a match on the pages in google and the logs i found a similarity! Only pages that was spidered by googlebot ip 64.68.82.66 are visible in Google!

Can anyone else confirm this?

Leif

12:24 pm on June 10, 2003 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:June 29, 2001
posts:2145
votes: 0


It is the freshbot. Do a site search for "freshbot" and you will find out what is happening.
I can tell that it is normal.
12:28 pm on June 10, 2003 (gmt 0)

Junior Member

10+ Year Member

joined:May 6, 2003
posts:171
votes: 0


You have a NEW site.... and it has 60.000 pages?
1:38 pm on June 10, 2003 (gmt 0)

Junior Member

10+ Year Member

joined:Jan 5, 2003
posts:86
votes: 0



Rugles, I know it's freshbot. The interesting is that only the pages that are requested from ip 64.68.82.66 (not the other freshbot ips) gets in the index.

Perplexed, Yes, it's dynamic.

Leif

5:05 pm on June 10, 2003 (gmt 0)

Full Member

10+ Year Member

joined:May 22, 2003
posts:278
votes: 0


The "newest" index page of one of my sites has been up in google for 3 to 4 weeks now. But my new internal pages which were updated with the index page show the old result/cache at google. This pattern shows freshbot behaviour, but that should only last for 1 or 2 days .. this has been there for 3 to 4 weeks. Can anybody explain this pattern? Or is this an indication that deepbot is crawling?
5:11 pm on June 10, 2003 (gmt 0)

New User

10+ Year Member

joined:May 1, 2003
posts:17
votes: 0


Wondering too on Fresh vs Deepbot behavior. My site is regularly crawled (every other day) by freshbots, but none of the pages are added. Cache has been refreshed on some of the main pages, but still have old cache for most of the site. Is this the freshbot doing some freshbot work (refreshing some cache) and otherwise doing deepbot work (just gathering pages for the "next" google index update?
12:49 pm on June 11, 2003 (gmt 0)

Junior Member

10+ Year Member

joined:May 21, 2003
posts:176
votes: 0


I'm still only seeing Freshie. No Deep crawl for the last 3-4 months for me. Even so, Freshie is only spidering the odd page here and there. I don't understand why my site gets so little attention, its over 3 years old and new content is added nearly every month. Maybe Google has penalized me for some obscure reason...
1:12 pm on June 11, 2003 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:Nov 4, 2002
posts:1687
votes: 0


I'm still only seeing Freshie

That's all that's been around for everyone since the April crawl, (except for one post I read by someone saying they had 216.239 around regularly... you had to wonder if they meant 216.39).

Maybe Google has penalized me for some obscure reason...

From reading posts here of people who actually were penalized, it seemed to me that google would continue crawling the site even if it were penalized. I'm not 100% on that, but the reasons for freshie not visiting often might be related more to your incoming links, PR etc.

No Deep crawl for the last 3-4 months for me

That seems odd... you should have had at least two deepcrawls in that time.
1:35 pm on June 11, 2003 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:Nov 4, 2002
posts:1687
votes: 0


Pricey, you show only 4 backlinks, one from your own site,(grey-barred), one DMOZ, (cat is PR5 but with a lot of links on it), one Google directory, (PR4), and a linking firm that I can't quite figure out...

You need more incoming links and to be sure of what that link company is doing.

Added: you might consider shortening the title somewhat, though I doubt that that is your problem.

5:30 am on June 13, 2003 (gmt 0)

Senior Member

WebmasterWorld Senior Member googleguy is a WebmasterWorld Top Contributor of All Time 10+ Year Member

joined:Oct 8, 2001
posts:2882
votes: 0


Hey, I think that a lot of people have noticed that freshbot is crawling deeper into websites where the deepbot used to explore.
5:35 am on June 13, 2003 (gmt 0)

Preferred Member

10+ Year Member

joined:Mar 20, 2003
posts:390
votes: 0


Hint, hint...
This 211 message thread spans 8 pages: 211
 

Join The Conversation

Moderators and Top Contributors

Hot Threads This Week

Featured Threads

Free SEO Tools

Hire Expert Members