Welcome to WebmasterWorld Guest from 54.144.68.27

Forum Moderators: goodroi

Message Too Old, No Replies

Robots.txt

Huh?

     

zoobie

6:37 pm on Jun 19, 2003 (gmt 0)

10+ Year Member



My index is just a menu using only javascript buttons with the rest of the content in an iframe. No pages are linked to each other via <a href>.
Now, regarding the robots.txt file, if I tell the robots to spider the whole site, will they be able to do this even though the pages themselves are not linked with any <a href>?
...or should I just use a meta robots tag?
Thx

Macguru

6:52 pm on Jun 19, 2003 (gmt 0)

WebmasterWorld Senior Member macguru is a WebmasterWorld Top Contributor of All Time 10+ Year Member



Hi zoobie,

Neither of them will help. They are exclusion standards, not inclusion aids.

Do anything you can to get real links.

Noscript, noframes or site map if you like, but you definitly need links.

 

Featured Threads

Hot Threads This Week

Hot Threads This Month