Blocking Yahoobots from Cache-ing

[internetheaven]

Wondered if anyone knew how to alter a meta tag for just Yahoo so that if it hits a page the "no-cache" meta tag is inserted?

I can do it based on referrer, GeoIP location and much more but I'm having trouble finding anything on php-ing for crawlers.

Yahoo always caches my pages wrong and anyone that clicks the "cache" button in the Yahoo search listings gets a really messed up page.

Any help would be appreciated
Thanks
Mike

[RonPK]

You could look at the user agent string:

<?php 
if ($_SERVER['HTTP_USER_AGENT'] == 'Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp)') { 
 echo '<meta [...]>'; 
} 
?> 

[edited by: eelixduppy at 3:06 pm (utc) on Dec. 4, 2007]
[edit reason] delinked code [/edit]

[PHP_Chimp]

Have a read of [help.yahoo.com...]
As you can use the info on that page in combination with the post above to give you what you want.

[internetheaven]

Thanks RonPK, do you think it would still work if it was just:

if ($_SERVER['HTTP_USER_AGENT'] = 'Slurp')

just incase Yahoo are using any variations?

Thanks
Mike

[eelixduppy]

You can do something like this:

if ([url=http://us2.php.net/stri-pos]stri_pos[/url]($_SERVER['HTTP_USER_AGENT'],'slurp')!== FALSE)

[internetheaven]

You can do something like this:

if (stri_pos($_SERVER['HTTP_USER_AGENT'],'slurp')!== FALSE)

That causes a 500 server error.

Mike

[PHP_Chimp]

if (stripos($_SERVER['HTTP_USER_AGENT'], 'slurp')!==false) {

stripos [uk3.php.net], must have been a typo

What are you doing within that if block?

[edited by: PHP_Chimp at 7:54 pm (utc) on Dec. 6, 2007]

[eelixduppy]

oops, nice catch. That's what happens when do things from memory; you add random underscores in there :)

[PHP_Chimp]

Np, I wish php would decide if there are going to underscore between every word, or not. It is a pain trying to remember if stripos or str_split (or any of the other functions) have or have not got there _'s...The quick search box in the manual is a life saver :)

While working away last night I was pondering your 500 error from the code above. As when you say you are getting a 500 error are you putting this code directly into the htaccess file? As php wont work in there.

You could use -
RewriteCont %{HTTP_USER_AGENT} slurp [NC]
RewriteRule (.*) $1 [L, E=agent:yahoo]

This should set the environmental variable $_ENV['agent'] = 'yahoo' for anything identifying itself as slurp. You could then check for this variable on every page and if it is present rewrite your tags. The example above assumes that you are not passing everyone through a single page, however you could always change the $1 to be whatever page you were using for checking for slurp.
I havent used htaccess to set environmental variables for quite a long time, so you may well need to check this code with someone who is a bit more up to date with that sort of thing.