Forum Moderators: goodroi

Message Too Old, No Replies

duhhhhh, dumb blonde needing help with robot.txt

I don't have a clue - go easy please....

         

ladyluck

5:36 am on Oct 9, 2004 (gmt 0)

10+ Year Member



Hey guys!

I have read the past post, re-searched the web and still have no clue as to how I add a robot.txt. file to my web site to prevent my images from being displayed in yahoo (and other search engines)

1. I copy and pasted the following:
User-agent: *
Disallow: /images/

I saved it as robot.txt and uploaded the text file to homestead.com (that's where my domain is located, through homestead.)

That didnt work. (you already knew that right!?)

2. Next, I read that you could by-pass the robot.txt thing and add a: HTML META tags.

I logged into my site at homestead and added the HTML file of: <META NAME="ROBOTS" CONTENT="NOINDEX">
on each page that contained images that I didnt want taken by the robots.

....that didnt work either.

My questions are (please hang with me on this, I am so new to all of this stuff and at wits end)

--I copy/paste the robot.txt file and upload it to where? so it will appear where?

--Does it have to be on the MAIN PAGE?

(I own a domain and just use homestead for the web space) so, do I add the .txt file to: [mydomain.com...] in the address line?

--will that mess up all the links to my site? If I changing the [address...] and add the .txt file to the end?

OR------Can I do the <META NAME="ROBOTS" CONTENT="NOINDEX"> and have that work successfully? if so, how and where do I place that?

I know, I know, it's a lot to ask! but you guys have been there and done that and I am clueless when it comes to this robot stuff. Please help....

It's important to me to get the robot.txt file in the proper place to prevent anyone having access to my images.

Thank you - Thank you - Thank you! for any help you can provide. remember....computer learner here, go slow with me....real slow in explaining! I ain't a pro like you guys are.

Thanks again!

deejay

5:40 am on Oct 9, 2004 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



filename should be robots.txt, rather than robot.txt for a start :)

It doesn't need to be linked from anywhere on your other web pages, just place the file in the root of your domain, eg, www.mydomain.com/robots.txt, and spiders will request the file automatically.

I'll leave your more specific questions for the next member along. :)

ladyluck

6:11 am on Oct 9, 2004 (gmt 0)

10+ Year Member



Thanks for your reply.

So, I created the file robots.txt and uploaded it to my webpage using FTP...I placed it in the domain's files. I thought sure I did it right, but no luck....

Then I typed in the www.domainname/robots.txt and this is what appeared:

# HomesteadSpider allowed to crawl everywhere
User-agent: HomesteadSpider
Disallow:

# Other bots blocked from the core Homestead site
User-agent: *
Disallow: /~site/Scripts_ElementMailer
Disallow: /~site/Scripts_ExternalRedirect
Disallow: /~site/Scripts_HitCounter
Disallow: /~site/Scripts_NewGuest
Disallow: /~site/Scripts_RealTracker
Disallow: /~site/Scripts_Track
Disallow: /~site/Scripts_WebPoll

That's not what I saved as a txt file! how can I alter an exsising robots file provided by homestead?

CAN I over-ride it or alter it? I just want to prevent images from being plastered all over the search engines.

Thanks again! :-)

moltar

6:37 am on Oct 9, 2004 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



You cannot edit homestead's robots.txt. You can only have one robots file per domain. Either get your own hosting, or you are out of luck. Really, not that many options :)