Welcome to WebmasterWorld Guest from 100.26.182.28

Forum Moderators: Robert Charlton & goodroi

Message Too Old, No Replies

Can Googlebot see through mod_rewrite?

     
7:09 pm on Feb 27, 2005 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:Apr 2, 2002
posts:1099
votes: 0


Hi, I did some mod_rewrite to make some URLs SE-friendly. I was wondering if Googlebot will treat my mod_rewrite'ed URLs as normal non-redirected URLs or redirected URLs (301, 302...)

Thanks!

7:34 pm on Feb 27, 2005 (gmt 0)

Senior Member

WebmasterWorld Senior Member jdmorgan is a WebmasterWorld Top Contributor of All Time 10+ Year Member

joined:Mar 31, 2002
posts:25430
votes: 0


That depends entirely on how you wrote your rules. If you wrote them to do external redirects (using the [R] flag or a canonical URL), then they're visible. If you wrote them to do server-internal rewrites (using only a local URL-path and no [R] flag), then they are not visible.

There's nothing mysterious about a search engine spider; It fetches URLs just like a browser. If the effects of a RewriteRule are visible to a browser -- that is, if the address bar updates to show the new URL -- then a spider will also see the redirect, unless you are cloaking.

Jim

7:51 pm on Feb 27, 2005 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:Apr 2, 2002
posts:1099
votes: 0


Thanks for your prompt reply.

In my mod_rewrite, I only have this.
RewriteEngine ON
RewriteRule ^(.*)$ index.php?page=$1 [QSA,L]

It's not R... so what do you think?

8:14 pm on Feb 27, 2005 (gmt 0)

Preferred Member

10+ Year Member

joined:Mar 5, 2004
posts:433
votes: 0


Google won't notice, this is entirely server side.

You can verify this by inspecting the headers and content (e.g. with Firefox web developer extensions, Sam Spade's safe web browser or wget -S on the Linux command line).

8:17 am on Feb 28, 2005 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:Feb 25, 2003
posts:2528
votes: 0


I would imagine that Googlebot could make an educated guess. The header information shows "chunked" on most occasions instead of an actual file size - surely that's an indication that the file it is retrieving is dynamic, not static?
9:06 am on Feb 28, 2005 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:Apr 2, 2002
posts:1099
votes: 0


internetheaven,

I'm sorry... could you elabroate? I don't entirely understand what you are trying to say.

Chunked? ...?

9:45 am on Feb 28, 2005 (gmt 0)

New User

10+ Year Member

joined:Feb 24, 2005
posts:9
votes: 0


On my website there is no "chunked" header, I force content-type: text/html. Still there is no content-length header. How do you think the google spider will consider such pages?
 

Join The Conversation

Moderators and Top Contributors

Hot Threads This Week

Featured Threads

Free SEO Tools

Hire Expert Members