Welcome to WebmasterWorld Guest from 54.166.191.159

Forum Moderators: open

Message Too Old, No Replies

Are all crawled pages in the *live* index?

Is FreshDeepBot putting all new pages into the live index for good?

     
11:43 pm on Jul 14, 2003 (gmt 0)

10+ Year Member



So, have people noticed new content which has been crawled, but not posted to the live index?

Just trying to figure out if we're going to see a July update, or if what you see is what you get.

6:06 am on Jul 15, 2003 (gmt 0)

WebmasterWorld Senior Member marcia is a WebmasterWorld Top Contributor of All Time 10+ Year Member



We have no way of knowing whatever will happen again. We can only guess, which is fruitless and will drive us crazy.

I suspect new pages are getting in if they're crawled if they have enough PR, and if they don't have enough they won't. Just a suspicion, no basis in fact.

6:36 am on Jul 15, 2003 (gmt 0)

10+ Year Member



For my keywords the results from www-cw are much different than from the other datacenters.

I'm wondering if this old data or new data. And if it has something to do with an update.

I need an update! I do not like this deep/fresh continuous update thing at all!

What about the backlinks? I guess we need an update for our backlinks. I have received a lot of links in the last weeks. They aren't there yet. So, I still have hope for an update.

7:47 am on Jul 15, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



I have a bunch of pages that have been crawled but aren't live yet, but I also have pages that have been added gradually over the last several weeks.

I'm seeing many differences between the datacenters. Some have more pages of my site, but others have newer snapshots.

I need an update! I do not like this deep/fresh continuous update thing at all!

I think we'll all love it, once we see that it's working.

9:14 am on Jul 15, 2003 (gmt 0)

10+ Year Member



Deepbot has been picking files up from me but no sign of them being catalogued yet.
9:56 am on Jul 15, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



I have had a bunch of pages that were crawled (on the 5th) and updated (around the 9th?) and are now stable and happily bringing us traffic ;]

However, our index page has a fresh tag in some serps, no fresh in others. Never noticed this before...

11:24 am on Jul 15, 2003 (gmt 0)

10+ Year Member



I just see that another 10 files have entered the index in
the past few hours. I have freshbots around all the time but I think these are due to the current deepbots. (crawlers/crawls).
11:43 am on Jul 15, 2003 (gmt 0)

10+ Year Member



It is great to find out about site: ... -qwerrew and
allinurl: ... but is there anything equivalent for external
links. That would be invaluable too.
12:34 pm on Jul 15, 2003 (gmt 0)

10+ Year Member



I have a new website, freshbots has been on it for 3 days, last week, after that my web site appeared in google results. Sadly I have PR at 0, that's because backlinks haven't been calculated, yet.

What I think is that google is trying to index all pages as often as it can and later, maybe every month is calculating PR , backlinks and other stuff.

12:40 pm on Jul 15, 2003 (gmt 0)

10+ Year Member



loganoski,

I'd agree with that, that was the reason they started with the freshbot in the first place and I believe GG said it was just in testing phase a while back, this is just the next step in the plan for freshbot in my opinion, they can't go recalculating PR and backlinks every day but they want to keep more and more content fresh, which is better for us all.

Cheers,
Nigel

2:25 pm on Jul 15, 2003 (gmt 0)

10+ Year Member



For us, pages have been updated, but the cache for the index has only been updated with a refresh every few days and then it goes back to the older cached version. There still needs to be a hard update, imo.
3:21 pm on Jul 15, 2003 (gmt 0)

10+ Year Member



I was wrong about an extra ten pages being entered in the index. Sorry about that.
7:00 pm on Jul 15, 2003 (gmt 0)

WebmasterWorld Senior Member g1smd is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



Getting freshed every single day for the last few weeks, and changes to pages are being cached straightaway. Not reverting to old versions at all.
7:25 pm on Jul 15, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



How high is the PR on that site G1?
7:56 pm on Jul 15, 2003 (gmt 0)

10+ Year Member



Last month the dance started on the 15th.

Looks like there may not be a 'dance' in July at this rate.

If there ever is a dance, what is going to be the new name? =)

jd

8:20 pm on Jul 15, 2003 (gmt 0)

WebmasterWorld Senior Member g1smd is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



>> How high is the PR? <<

Has always been white bar 0/10.

Site at this new domain went live 2003-05-04 and first appeared in Google on 2003-06-16, at the very same time as the old domain was dropped from the listings. The old domain had noindex,follow on all pages, since late May, and all "internal" navigation of the old site was modified to point to the equivalent page on the new site instead. Old site still shows PR3 even though it is nowhere in the index.

4:37 am on Jul 16, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



Hi,

Sometimes your fresh file is index but it still display the old *deep crawl index. Especially when you change your title.

SO you need to check your cache to see if the fresh bot got the files into the index.

5:32 am on Jul 16, 2003 (gmt 0)

10+ Year Member



Hello,

My site is a PR 1 and is getting fresh tags daily for about 30 to 60 pages out of 250 in site. Cache for these pages is 216... Problem for me is that many of these pages are dropped from the index once the fresh tags are gone.

I am tired of pasting all new links onto my index page (as one poster suggested) just to entice deepfreshy to crawl them. I don't see much benefit in sticking new links on my index page as deepfreshy's crawling doesn't mean that new pages will stick sans fresh tags.

Marcia may be right regarding her hunch that higher PR pages may have more stickiness in the index.

12:00 pm on Jul 16, 2003 (gmt 0)

10+ Year Member



How do I get freshbot to my site, I'm currently indexed, but have updated the site to try to optimise it, but i'm not being crawled?
1:49 pm on Jul 16, 2003 (gmt 0)

10+ Year Member



Well now I can confidently report that the number of files
indexed at my site has gone up from 232 to 256 out of 340
plus 20 graphics files. Whether this was from freshbot or deepbot (both are around) I dont know.
2:34 pm on Jul 16, 2003 (gmt 0)

10+ Year Member



Does anybody doubt that there will have to be some type of a regular update just for the purpose of calculating the new PR? The iteration process just seems to complicated to do on the fly. I would think that computing anchor text would be possible on the fly, but even that may be best left to a once a month adjustment.
2:35 pm on Jul 16, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



Hmm, I bet that everything could be done gradually. For example, the index could keep track, for each page, when its PR was last updated. Then, whenever there was time, it could find the page with the most stale PR, and recalculate it based on a database of backlinks.

But, it wouldn't suprise or bother me if PR and anchor text was done monthly. I'd rather see new pages appear in the index quickly, than to have their PR be up-to-the minute.

2:58 pm on Jul 16, 2003 (gmt 0)

10+ Year Member



Silly me. I am slow-witted. I checked my files and none have the date against them so I assume that the newly-indexed files came from deepbot rather than freshbot. The number indexed has now dropped from 256 to 255 but that still means twenty+ more have been indexed since this morning.
4:12 pm on Jul 16, 2003 (gmt 0)

10+ Year Member



I have pages in the index with no date with content from 4 days ago (not new pages just new content), I would not trust the date to mean they are from the deep index.

Cheers,
Nigel

4:59 pm on Jul 16, 2003 (gmt 0)

10+ Year Member



Hi, all.

I lost my major entry page from Google's listings.
I don't know what happened to it.

It used to be the next highest PR page(PR5-after home page) or the highest ranked page for my sites with most my keywords.

It is now at the bottom of my hudreds pages on Google.

When I clicked the cached version, it only showed half of the page with Broken Html code.

Is it because there might be an interruption during the crawls?

Every other pages look normal.

How can I give a weight to this page back?
Do I have to wait a month or so?

It is causing Adsense ads problem too. it is showing public ads.

I believe Googlebot or Mediabot don't understand what this page is about.

Any help will be appreciated.

5:58 pm on Jul 16, 2003 (gmt 0)

10+ Year Member



My Home Page has 15 July against it so Freshbot must have taken a look. It hasnt been updated for a few weeks.

The pattern I am getting from Freshbot and Deepbot makes me doubt some of the claims being made for FreshDeepbot.
Although I have had nothing like the number of Deepbots I had around three months ago when I moved my site to a new place and had to start again.

6:21 pm on Jul 16, 2003 (gmt 0)

10+ Year Member



If GoogleGuy is correct then this should be the last time we see Deepbot. But it is certainly still active.
 

Featured Threads

Hot Threads This Week

Hot Threads This Month