is it necessary for dynamic pages

aspx pages

[qais77]

Hello,

I am new to this, am running online shopping website and it's dynamic pages aspx, is it necessary to make robots.txt file and please give me some tips. waiting for your reply.

thanks alot,
Qais

[ThomasB]

Qais, first of all welcome to WebmasterWorld!

It's actually pretty hard to answer your question as there can be a lot or no problems at all with dynamic sites. Generally speaking you should look out for the following:
- Stop bots from crawling duplicate content (printer friendly, other sort option, ...)
- Stop bots from crawling private information (mysql-DB, pwds, ...)
- Stop bots from accessing order-forms and submitting them

These are imho the most important things you should take care of when creating a robots.txt file

[qais77]

Hi Thomas,

Thanks for your prompt reply. Actually, I did the same what you mentioned, hope it will work well with me.

thanks