Forum Moderators: goodroi
I am brand new to this site it has a wealth of info but I am still lost. I have read many postings about robot.txt files recently and must admit I am a bit confused about how to configure mine properly. First of all I have a mac comp I don't believe I have notepad. I don't even have a word processing prog at all yet. I can't just cut and paste an example of this site then and just use it somehow?
So robot.txt is not something I put into the metatag or anywhere into the webpages on my site at all right? I need to somehow make one and then upload it. That is where I am confused. I know I put it in the root file on the local side of things but if I'm understanding this I don't have to send it to the remote side?
Also I am a photographer my site is image ridden. I do want to take that into account when implementing my robot.txt file. I want the exposure with the spiders to index my site but if you were a photographer considering the circumstances and value you place on your photos what might your robot.txt file look like?
Any info appreciated thanks
This should at least help with imagebots adding my pics to image search engines correct.
When I put that into a tag is that allowing robots to crawl everything else then? That would be ideal. Everything but my images.
Thanks
In Dreamweaver, make a new file, switch to code view, delete everything and type in your robots.txt instructions. From your post above that file would read:
useragent: *
disallow: /images
Make sure there is an extra return at the end! Save as... "robots.txt" in the same folder your website's main page (index.html) exists, then upload the file. It's more complex with TextEdit because you have to select "Make Plain Text" from one of the menus before you type the code but otherwise it's the same.