Welcome to WebmasterWorld Guest from 54.161.84.51

Forum Moderators: goodroi

Message Too Old, No Replies

Robots.txt

Huh?

     
6:37 pm on Jun 19, 2003 (gmt 0)

Junior Member

10+ Year Member

joined:Nov 20, 2002
posts:179
votes: 0


My index is just a menu using only javascript buttons with the rest of the content in an iframe. No pages are linked to each other via <a href>.
Now, regarding the robots.txt file, if I tell the robots to spider the whole site, will they be able to do this even though the pages themselves are not linked with any <a href>?
...or should I just use a meta robots tag?
Thx
6:52 pm on June 19, 2003 (gmt 0)

Senior Member

WebmasterWorld Senior Member macguru is a WebmasterWorld Top Contributor of All Time 10+ Year Member

joined:Dec 30, 2000
posts:3300
votes: 0


Hi zoobie,

Neither of them will help. They are exclusion standards, not inclusion aids.

Do anything you can to get real links.

Noscript, noframes or site map if you like, but you definitly need links.