Forum Moderators: open

Message Too Old, No Replies

Does a Freshbot Crawl Get You Listed

Or is this an artifact of the current update cycle?

         

jonrichd

11:02 pm on Mar 3, 2003 (gmt 0)

10+ Year Member



I have a new site that went live on 2/28, on 3/2 I got visited by Freshbot, with another visit today (3/3). There is only one link to this site, and the home page of the linking site has a fresh tag of 3/2, but no tag for the interior page that contains the link.

The new site does not appear to be visible in the SERPs.

Is this normal, or is it related to the update cycle?

I have a third site that is already in the main index, and has had plenty of freshbot visits to multiple pages, but I've never seen a fresh tag for it, either.

jonrichd

1:10 am on Mar 4, 2003 (gmt 0)

10+ Year Member



I may have somewhat of an answer to my own question.

While doing some other searching I found a freshbot-dated page of 3/2. This page is a news page from a radio station. When I looked at the Google cache, the cache page was dated Wednesday, 2/26, so there seems to be a delay between when Freshbot visits, and when the page gets displayed.

jdMorgan

5:33 am on Mar 4, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



jonrichd,

A freshbot crawl may get you listed, but usually it's only a temporary listing. Unless your page is linked-to by high PageRank pages and is updated often and significantly, these freshbot listings typically only last from one to three days.

After the next index update, your site is likely to get deep-crawled. The results from that deep-crawl will then appear during the next dance a few weeks later. You may be able to keep your pages showing in the index by pleasing the freshbot as descibed above, but most new pages tend to pop in and out of the index for a month until the results of their first deep-crawl are integrated into the "permanent" index. The frustration of watching a new page pop in and out of the index, and of waiting for the update cycle are two of the things that contribute to the update fervor here on WebmasterWorld. Despite the fact that most of us have been through it many times, the waiting for a new page to get solidly indexed never gets much easier.

Best,
Jim