homepage Welcome to WebmasterWorld Guest from
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member

Home / Forums Index / Google / Google News Archive
Forum Library, Charter, Moderator: open

Google News Archive Forum

How to Attract Freshbot....
Getting freshbot to start making the rounds again...

 8:12 pm on Aug 15, 2003 (gmt 0)

Ok all googlers...

As noted in another thread, I have the theory that part of the disappearing-index page problem so many are seeing (including myself) has to do with lack of freshbot visits.

Specifcally, I have a PR6 site that, despite almost daily deep-crawling by various googlebot's, frechbot has not hit my front page in months, or if it has, has failed to give me a fresh tag in over 6 months.

Now, this particular site WAS receiving upwards of 3000+ total unique visitors per day (about 70% from google) with over half hitting the front page first.

However, since this mysterious glitch (disappearing index) has popped up, traffic is cut in half, with most visitors finding the site via an internal content page rather than via the front page (which has not made my advertisers very happy).

In watching this phenom occur over the last 3 weeks or so, and most recently ab out a week ago, it appears to me that those sites that, for whatever reason, are failing to attract freshbot to their front pages are slipping from the serps for the primary targeted keyword/phrase.


The big question is, how do I get frechbot to strat coming back around on daily basis, and giving me a minty-fresh date?

In my efforts to do just that, here's what I've been doing for the past week or so:

1. New content is added to site daily (this was occuring anyway, with over 7000+ pages currently indexed)

2. Small changes to note new content are added to index page daily, including an "Last updated on..." reference near the top of the page.

Other questions...

What else should I be trying?

What prevents/influences frechbot to stop visiting a site (that continually adds content) on a daily basis?

Why does freshbot continue to visit small (spammy) 9-10 page sites that rarely (if ever) change their content, yet it fails to hit our front page?

Why are there 3-4 googlebot crawlers hitting our site almost everyday (including right now), but none of them seem to want to grab the PR6 index page?



 10:37 pm on Aug 15, 2003 (gmt 0)

I can send Googlebot your way if you like. Just sticky me the URL...


 10:43 pm on Aug 15, 2003 (gmt 0)

Personally I think freshbot only visit pages that are linked from a freshbotted page.

Soo - I would look for links from pages that are getting hit from freshbot themselves and hopefully the bot will have enough left to visit your pages too.

I launched a site in Feb and had freshbot every day or every other day until recently and now I only see Googlebot occassionally (although a bit more over the last few days - but in a deepcrawl fashion).

Anyway - I am sure more links,esp from freshed sites, will bring freshbot back.


<edit>Silly Typos changed - it is late UK time and I am depressed and had a few to many glasses of wine ;)</edit>

[edited by: Dayo_UK at 10:46 pm (utc) on Aug. 15, 2003]


 10:43 pm on Aug 15, 2003 (gmt 0)

Same here...if you like...will get the little devil to visit you


 6:54 am on Aug 16, 2003 (gmt 0)

As already mentioned, the easiest way to get freshbot to visit is to get links from sites that are already visited regularly by freshbot. But that's probably still not a guarentee.

Is the index page of your site listed in DMOZ? If not, that woud be a good start.

1. New content is added to site daily (this was occuring anyway, with over 7000+ pages currently indexed)

With over 7,000 pages indexed, why is your index page recieving 50% of the referrels from Google?

Idealy you should be receiving traffic more evenly spread out from all over your site.

How many critical terms/phrases are your index page targetting?

Does your index page have a substantially larger PR than your top subpages?

My recommendation would be to target some of those critical keywords (that you're no-longer getting) with subpages.


 9:13 am on Aug 16, 2003 (gmt 0)

the easiest way to get freshbot to visit is to get links from sites that are already visited regularly by freshbot. But that's probably still not a guarentee.

I reckon I can send the freshbot to any URL I like within the space of about 24 hours.


 9:53 am on Aug 16, 2003 (gmt 0)

same here. Freshbot visits me on average 2500 times a day



 10:01 am on Aug 16, 2003 (gmt 0)

I have a fairly static PR6 page that was freshbotted on 15/8. It gets crawled almost daily, so I don't know if having fresh content has a lot to do with it. I might change something on that page about once a month, perhaps twice but sometimes it is static for months.


 10:49 am on Aug 16, 2003 (gmt 0)

Ok folks...

To test your theory, I now have 2 links for 2 different sites which have had freshbot tags on an almost daily basis. I'll let you know if it has any effect.


 12:49 pm on Aug 18, 2003 (gmt 0)

Hmmm...So much for that theory...

I've had front-page link to my site (with appropriate anchor text) up at two PR6 sites that have both received freshbot visits (along with new fresh-tags) over the last two days.

However, my site STILL has not had a freshbot visit (at least not to the index page).

Ideas anyone?


 11:50 pm on Aug 18, 2003 (gmt 0)

Google seems to update my home page link every two days, for many of my sites.

I assume this is freshbot, but the weird thing is that it generally updating only the homepage?

How would i make it dig deeper?

Global Options:
 top home search open messages active posts  

Home / Forums Index / Google / Google News Archive
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved