Multiple Hits from Freshbot on Same Internal Page

Google Freshbot Revisit Behavior

[nativenewyorker]

What does it mean when Google freshbot retrieves an internal page 2-3 times a day for 5 days in a row? Anyone ever see this? I notice the server keeps returning 200 codes for this page. In the meantime, Google is doing a deep crawl of the rest of my site and I only see freshbot on that one page of 200+.

Thanks,
Ted

[Brett_Tabke]

It means it likes that page. It's a good thing. Keep it updated and current.

[xerxes]

Nativenewyorker: are you changing that internal page several times a day with product updates or something similar? How many times on the average are you changing items on that page daily? Probably this is what Freshbot is responding to. The server gives the fresh 200 and Freshbot sees it, I think. Now, the question appears to be how many 200's (page changed and uploaded) are needed to trigger a Freshbot visit? Please give your thoughts. Thank you.

[nativenewyorker]

xerxes,

I haven't made any changes to that page in over a week, yet freshbot is still coming to that page daily. Maybe it has something to do with Google using multiple spiders. A different spider is requesting the page each time and referencing an old cache. That could explain the server codes "200". I'm at a loss to what else it could be. Believe me, I would be ecstatic if I could have freshbot crawl my entire site every day.

I'll check my logs tomorrow to see if it was the same IP requesting that page.

Ted

[xerxes]

OK, I pulled an all-nighter and here are a couple of tips: put into Google search: 64.68.82 googlebot 216.239.
and you get a LOT of site stats for today's stats! Not old stats, TODAY's! I found one to give me a recip link. Some of these guys have up to 9 visits LAST NIGHT from Freshbot! Our dream, huh? So I spent the night studying the layout of their index pages and their sites. Very interesting. I immediately modified our site accordingly. Try it and see what you find. You will truly be AMAZED! I will study our logs and see if Freshbot starts to respond to new layout.

[yetanotheruser]

nativenewyorker,

I've found out only recently (from this forum) that googlebot comes in two forms, deepcrawl and freshbot.. but I've seen multiple hits from freshbot in the logs on the same day before.

I always assumed that these multiple hits were due to googlebot following external links from another site. A kind of validating-bot as it were..

Infact, I am sure that I have seen it going through our clients sites, and that the hits on our site match the bot passing the pages that have our credit link on them..

Maybe I'm wrong.. Perhaps to get freshbot to come to your site you don't neccassarily have to update your content very often, but just be linked from a page that does update very often?

:) J.

[nativenewyorker]

xerxes & yetanotheruser,

Looking at the logs, Freshbot has been hitting the page anywhere from 1 to 3 times a day. The Googlebot IP was only repeated once in the logs. I'm not sure about anyone linking to one of my internal pages, but it is possible and worth further consideration.

yetanotheruser said:

Perhaps to get freshbot to come to your site you don't neccassarily have to update your content very often, but just be linked from a page that does update very often?

This is quite possible. I don't show any backlinks on that page yet, let's see what happens in a couple of weeks.

Ted