Forum Moderators: open
I began noticing this week, that any page Googlebot fetched started showing up within the next 48 hours in their regular searches and also the fresh cache of the page was there, too. At first I thought it was just a few pages that get the fresh bot every 24 hours, but upon further examination it's not freshbot pages, it's the regular crawl that's showing up the next day or so after Googlebot fetches the page.
I don't think I would have noticed this unless the pages were totally changed with new content and titles. I remember reading in the forum that people thought Google was updating information from their crawl each weekend. However, I'm starting to think that they have turned all their crawling machines into fresh bot crawlers and they are updating everything they crawl every 24 hours. I've reviewed my raw log files and have found all pages that are fetched on one day, showing up in regular searches the next.
I would sure like to hear from some of the more experienced people here and see what they think. Also, it would be great to hear from folks that have completely changed their websites recently to see if this might be happening to them, too.
Anyone else notice this?
The other thing I noticed is if I put up a new page and someone higher ranking than me links to it Googlebot will stop by and grab the page within a couple hours. It appears in the index within a day or two.
There's no problem, all bots are now deepfresh crawlers. Have been for quite few weeks now. My fresh pages are picked up daily (in thousands) as well as old pages revisited and updated. I found that googlebot mostly takes only root (/) every day on the sites that have been crawled before. When I make a new site it gets spidered in full the same or next day though.
My guess would be: google continuously updates fresh and old pages with changes visible in index within 48 hours. And every so oftern the crawlings decrease to give google space to recalculate PR, backlinks, etc..
Darko
Backlinks and PR came up quite nicely for most of the old and even the newer sites in the past 2 weeks, although I haven't seen any real SERPs benefits from the added PR yet. (I also haven't seen any evidence of 'weekend' PR changes on the several dozen sites I manage).
The newer sites that are less than 3 months old seem to be more problematic recently. Even with a little 'nudging,' it takes almost 3 weeks to get in the index, then they float in and out every few days.
Others will have much more on this, but for me, the older sites are about the same in SERPs and traffic, and it seems the newer sites are much less stable than in previous months.
During the Dominic..Esmerelda...Rolling Update thing I was busy adding thousands of pages to one of my sites. (None of them have been added to the index). While checking to see what files of mine were actually in Google I ran across [mywebhost.com...] in the SERPS. Remember that these files have been deleted from the server for NINE MONTHS. I still had the original files so I recreated the directory and uploaded all of the files to the server. Guess what? [mywebhost.com...] is NOW in #1 position in the SERPS for "widgets".
Files that should have long "faded out of the index" because they were DELETED turn up in the index 9 months later as "current". Once the DELETED files are restored to the server the index page NOW ranks #1 in the "current" (read as "pretty recent" if it makes you feel better) SERPS for the primary search argument. Current? Yeah,sure they are. The $64,000 question still remains. What about the 2,000+ pages that I have added to my site over the past month and a half? How long before THEY are "current" or even "pretty recent"?
