Can Googlebot see through mod_rewrite?

[irock]

Hi, I did some mod_rewrite to make some URLs SE-friendly. I was wondering if Googlebot will treat my mod_rewrite'ed URLs as normal non-redirected URLs or redirected URLs (301, 302...)

Thanks!

[jdMorgan]

That depends entirely on how you wrote your rules. If you wrote them to do external redirects (using the [R] flag or a canonical URL), then they're visible. If you wrote them to do server-internal rewrites (using only a local URL-path and no [R] flag), then they are not visible.

There's nothing mysterious about a search engine spider; It fetches URLs just like a browser. If the effects of a RewriteRule are visible to a browser -- that is, if the address bar updates to show the new URL -- then a spider will also see the redirect, unless you are cloaking.

Jim

[irock]

Thanks for your prompt reply.

In my mod_rewrite, I only have this.
RewriteEngine ON
RewriteRule ^(.*)$ index.php?page=$1 [QSA,L]

It's not R... so what do you think?

[arrowman]

Google won't notice, this is entirely server side.

You can verify this by inspecting the headers and content (e.g. with Firefox web developer extensions, Sam Spade's safe web browser or wget -S on the Linux command line).

[internetheaven]

I would imagine that Googlebot could make an educated guess. The header information shows "chunked" on most occasions instead of an actual file size - surely that's an indication that the file it is retrieving is dynamic, not static?

[irock]

internetheaven,

I'm sorry... could you elabroate? I don't entirely understand what you are trying to say.

Chunked? ...?

[mtishetsky]

On my website there is no "chunked" header, I force content-type: text/html. Still there is no content-length header. How do you think the google spider will consider such pages?