Forum Moderators: mack

Message Too Old, No Replies

robot.txt

how do I know if I need robot.txt file

         

Doug_R

5:31 am on Nov 10, 2003 (gmt 0)



I am new to marketing online, I have submitted to many engines and have been indexed (so they say) with some of them but can't find my site when i use thier search engine, even when I search the exact page title that is on my title page in thier search bar I still come up empty. the only way i can find my site is thru the address bar,using my whole url does the robot.txt file have something to do with my url were it is not indexing my site? how do i find out what my problem is and how to fix it?

Could someone please Help Me

Mohamed_E

1:34 pm on Nov 10, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



Welcome to WebmasterWorld, Doug_R!

The only function of a robots.txt file is to keep robots out of part (ot even all) of your site. Since that is not what you want to do, you do not need one.

The problem may be that you have a bad robots file, if so you should either remove it or replace it with an empty file.

There are many other possible reasons for not being indexed. Which engines did you submit to, and how long ago? Did you pay or free submission? More information will help us help you :)

Rumbas

1:47 pm on Nov 10, 2003 (gmt 0)

WebmasterWorld Administrator 10+ Year Member Top Contributors Of The Month



Hi Doug. Welcome to WebmasterWorld.

In fact we have a whole forum dedicated to Robots.txt, right here:
[webmasterworld.com...]

There you'll probably find exactly what you're asking for.