Forum Moderators: open
As noted in another thread, I have the theory that part of the disappearing-index page problem so many are seeing (including myself) has to do with lack of freshbot visits.
Specifcally, I have a PR6 site that, despite almost daily deep-crawling by various googlebot's, frechbot has not hit my front page in months, or if it has, has failed to give me a fresh tag in over 6 months.
Now, this particular site WAS receiving upwards of 3000+ total unique visitors per day (about 70% from google) with over half hitting the front page first.
However, since this mysterious glitch (disappearing index) has popped up, traffic is cut in half, with most visitors finding the site via an internal content page rather than via the front page (which has not made my advertisers very happy).
In watching this phenom occur over the last 3 weeks or so, and most recently ab out a week ago, it appears to me that those sites that, for whatever reason, are failing to attract freshbot to their front pages are slipping from the serps for the primary targeted keyword/phrase.
Sooooo....
The big question is, how do I get frechbot to strat coming back around on daily basis, and giving me a minty-fresh date?
In my efforts to do just that, here's what I've been doing for the past week or so:
1. New content is added to site daily (this was occuring anyway, with over 7000+ pages currently indexed)
2. Small changes to note new content are added to index page daily, including an "Last updated on..." reference near the top of the page.
Other questions...
What else should I be trying?
What prevents/influences frechbot to stop visiting a site (that continually adds content) on a daily basis?
Why does freshbot continue to visit small (spammy) 9-10 page sites that rarely (if ever) change their content, yet it fails to hit our front page?
Why are there 3-4 googlebot crawlers hitting our site almost everyday (including right now), but none of them seem to want to grab the PR6 index page?
Soo - I would look for links from pages that are getting hit from freshbot themselves and hopefully the bot will have enough left to visit your pages too.
I launched a site in Feb and had freshbot every day or every other day until recently and now I only see Googlebot occassionally (although a bit more over the last few days - but in a deepcrawl fashion).
Anyway - I am sure more links,esp from freshed sites, will bring freshbot back.
Dayo
<edit>Silly Typos changed - it is late UK time and I am depressed and had a few to many glasses of wine ;)</edit>
[edited by: Dayo_UK at 10:46 pm (utc) on Aug. 15, 2003]
Is the index page of your site listed in DMOZ? If not, that woud be a good start.
1. New content is added to site daily (this was occuring anyway, with over 7000+ pages currently indexed)
With over 7,000 pages indexed, why is your index page recieving 50% of the referrels from Google?
Idealy you should be receiving traffic more evenly spread out from all over your site.
How many critical terms/phrases are your index page targetting?
Does your index page have a substantially larger PR than your top subpages?
My recommendation would be to target some of those critical keywords (that you're no-longer getting) with subpages.
the easiest way to get freshbot to visit is to get links from sites that are already visited regularly by freshbot. But that's probably still not a guarentee.
I reckon I can send the freshbot to any URL I like within the space of about 24 hours.
Darko
To test your theory, I now have 2 links for 2 different sites which have had freshbot tags on an almost daily basis. I'll let you know if it has any effect.
I've had front-page link to my site (with appropriate anchor text) up at two PR6 sites that have both received freshbot visits (along with new fresh-tags) over the last two days.
However, my site STILL has not had a freshbot visit (at least not to the index page).
Ideas anyone?
I assume this is freshbot, but the weird thing is that it generally updating only the homepage?
How would i make it dig deeper?
