Forum Moderators: open
The index pages are changed often in images and in size but to no joy.
Google is showing one of my pages before it was optomised. The page is 6 weeks old. Now new fresh dates are appearing on all web sites except this web site.
Some website have less backlinks and less pages but have the freshbot nearly everyday.
Any thoughts on how best to qualify for the fresh bot or is it again pot luck.
I think you can 'qualify' for freshbots IF:
Anybody else?
leo
>>a dynamic page could be tricky if you only update the database but not the php/asp file.
Why do you say that please? If the database has changed, the data has changed, and the returned file size will be different. You can use header tags like this to properly inform the search engine when things change:
header("Cache-Control: must-revalidate,proxy-revalidate,max-age=120,s-maxage=120");
header('Last-Modified: '.date('D, d M Y H:i:s', ((time()+(3600*8))-60)).' GMT');
Please read the Tricking Googlebot on a dynamic site [webmasterworld.com] thread. If it is possible to get the 'last modified date' information without loading the complete page, freshbot might not load the page anyway.
I believe that PR5 begins the freshbot cycle. I have a number of sites that are PR5 and they are freshbotted on a regular basis during the crawling cycle.
I have many sites with 0(grey),1,2,3 PR and they all get freshbot hits daily. Some of them don't even have backlinks (the grey one).
They are all dynamic pages (cgi / hashed urls).
The trick?
validators (last-modified etc.)
[w3.org...]
You can check your site's cacheability here:
[ircache.net...]
I wish GoogleGuy could explain more in detail the operations of the Fresh Bot.
All my pages are static html pages with more pages then my competitors and more links, but still it looks like its going to be another month before Google will change my page.
Would it not be better for Google to freshbot every main page (index.htm) of all the websites in the database to keep it fresh. Some sites have every page re-freshed whilst smaller sites get nothing.
Exactly. Make sure you declare everything that you should, make it real fresh content, and freshie will come.
You can even send the correct filesize of a dynamic file in the headers BEFORE the file is sent. Here is how in PHP-- you must put the page in to the buffer, read the size, then deliver the buffer.
<?
// BEFORE ANY PAGE CONTENT
ob_start('ob_gzhandler');
..... rest of page goes here .....
header("Cache-Control: must-revalidate,proxy-revalidate,max-age=120,s-maxage=120");
header('Last-Modified: '.date('D, d M Y H:i:s', ((time()+(3600*8))-60)).' GMT');
header('Content-Length: ' . ob_get_length());
ob_end_flush();
// HEADERS NOW INCLUDE PROPER VALIDATION INCLUDING CONTENT-LENGTH OF THIS DYNAMIC PAGE
// ADJUST THE DATE/TIME IN LAST-MODIFIED TO REFLECT YOUR TIME ZONE, THIS IS FOR PST (GMT-8)
?>
As a side note, I've noticed absolutely zero usefulness from a practical standpoint in Fresh Bot coming to visit, but people here treat it like it's the holy grail so I've learned to appreciate it as a sign of respect ;)
We also have clients with a variety of products that often change or add new products every week. Why make a customer wait a month for 'fresh' information if the information is setting there?
I can't really imagine many who wouldn't want fresh information indexed for their site.....
You can even send the correct filesize of a dynamic file in the headers BEFORE the file is sent. Here is how in PHP-- you must put the page in to the buffer, read the size, then deliver the buffer.
BTW: simply sending the last-modified header is only have the job
The idea behind if-modified-since is that you only send the content if it changed since the specified timestamp.
So always keep track of your created pages (e.g. by implementing caching , which is always a good idea).
Then respond to the request by either sending "304 Not Modified" (and a Date header) or "200 OK"
I too can confirm that sending the content-length is an important thing and can speed up things enormously (HTTP/1.0 persistant connections).
I believe, the more you stick to the standards the more google likes you :)
I can't really imagine many who wouldn't want fresh information indexed for their site.....
I guess that's true. My point was only that I've never once seen a jump in sales or even hits for that matter as a result of Fresh Bot. I suppose though that the hits I'm getting are more relevant then because it's more up-to-date. Don't get me wrong, I'll take Fresh Bot over no Fresh Bot any day of the week :)
