Log Analysis Question

Log Analysis

[jjdesigns4u]

I just downloaded Sawmill. I like it. I am new to this so I do not understand everything I am seeing.

I took a look at my error log and I am geeting a lot of errors looking for the robot.txt. I do not have that file. Who is looking for it? Should I have it?

I also took a look at the click trails and the second one on the list was the robot.txt

Can someone please fill me in. I though if I did not want to stop spidering on a certain page I do not need the robot.txt

Thansk in advance!

[creative craig]

Even though you dont have the robot.txt file most spiders will still request it. The information on the txt will say if you want them to visit you and crawl your site and index your pages in their database.

If you have a look in [searchengineworld.com...] you will find so much information your head will explode :)

[jjdesigns4u]

I checked out that site. It had everything I was looking for. Thanks.

I don't think my head exploded......?