homepage Welcome to WebmasterWorld Guest from 50.19.172.0
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member

Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt
Forum Library, Charter, Moderators: goodroi

Sitemaps, Meta Data, and robots.txt Forum

    
Robots.txt
Huh?
zoobie

10+ Year Member



 
Msg#: 25 posted 6:37 pm on Jun 19, 2003 (gmt 0)

My index is just a menu using only javascript buttons with the rest of the content in an iframe. No pages are linked to each other via <a href>.
Now, regarding the robots.txt file, if I tell the robots to spider the whole site, will they be able to do this even though the pages themselves are not linked with any <a href>?
...or should I just use a meta robots tag?
Thx

 

Macguru

WebmasterWorld Senior Member macguru us a WebmasterWorld Top Contributor of All Time 10+ Year Member



 
Msg#: 25 posted 6:52 pm on Jun 19, 2003 (gmt 0)

Hi zoobie,

Neither of them will help. They are exclusion standards, not inclusion aids.

Do anything you can to get real links.

Noscript, noframes or site map if you like, but you definitly need links.

Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved