Forum Moderators: goodroi
[example.com...]
is written statically as:
[example.com...]
-----------------------------------
Options +FollowSymLinks
RewriteEngine on
RewriteRule form/(.*)/(.*)/$ /form.php?$1=$2
---------------------------------------------
Now on the inner page: [example.com...]
I have some links such as:
[example.com...]
These links I dont want crawled and are not rewritten into static pages.
My robots.txt file is defined as below:
User-agent: *
Disallow: /login.php
Disallow: /form.php
Disallow: /logout.php
Disallow: /signup.php
Disallow: /pwdremind.php
Disallow: /contact.php
Disallow: /priv_policy.php
Disallow: /disclaimer.php
Disallow: /terms_use.php
I was wondering if by disallowing form.php I will not be crawled on my statically created pages like:
[example.com...]
Will the above page be crawled by all robots?
[edited by: ThomasB at 2:58 pm (utc) on May 3, 2005]
[edit reason] example [/edit]