Drupal Specific: Search Engine's Dont' Cache Design

Search Engines only cache text file and not the theme

[Erku]

Hello,


When we do site:mysite.com in google or in other search engines we see that they don't cache the design/theme, but just the text version of it.

I can't bring a specific site example because rules here don't allow, but what can we do about it?

How can I fix this problem. Any insight and help will be greatly appreciated.

Could this problem cause huge drop in traffic?

Thank you for all your help.

[ergophobe]

It's just the cache right? If it looks crappy in the cache, won't they click through to your site? Maybe there's a side I'm not getting, but I don't see how that would affect your traffic.

That said, look in the cache and look at view source. What's happening?

Normally it should set <base href=> and then it should parse things out from there. In my drupal installs, it works fine (i.e. it pulls the HTML from the cache and then correctly resolves the stylesheets and even the favicon correctly, pulling those straight from my server).

So it could be that the base href is messed up or something like that.

Save the cached page locally and see what you need to change to get it to render correctly (that is, to pull the resources from your site) and that will give you some idea.

[BradleyT]

Are you sure it's not a browser issue?
[webmasterworld.com...]

Or another thought is that you're using aggregate CSS caching and the current version on the site is "ahead" of the Google cache version so your page looks unstyled. I had that issue in 5.x and it would just take a few days for Google to look for the new stylesheet name.

[ergophobe]

>>aggregate CSS caching

Good theory.

For those who don't know drupal, drupal has a setting that allows you to aggregate all CSS files into one big file, that then receives a random hash as the filename. Change your CSS and it regenerates the cache with a new hash and the old file then goes 404. That's not normally an issue, but the Google cache version can still be looking for the old file. Thus, unstyled content.

I suppose it wouldn't hurt anything to do like Yahoo does for it's YUI stuff and always rename files, never replace. So your cached CSS would get recreated and all current pages would reference it, but the Google cache would still find the original.