homepage Welcome to WebmasterWorld Guest from 54.226.235.222
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member

Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt
Forum Library, Charter, Moderators: goodroi

Sitemaps, Meta Data, and robots.txt Forum

    
Robots.txt
Huh?
zoobie




msg:1527334
 6:37 pm on Jun 19, 2003 (gmt 0)

My index is just a menu using only javascript buttons with the rest of the content in an iframe. No pages are linked to each other via <a href>.
Now, regarding the robots.txt file, if I tell the robots to spider the whole site, will they be able to do this even though the pages themselves are not linked with any <a href>?
...or should I just use a meta robots tag?
Thx

 

Macguru




msg:1527335
 6:52 pm on Jun 19, 2003 (gmt 0)

Hi zoobie,

Neither of them will help. They are exclusion standards, not inclusion aids.

Do anything you can to get real links.

Noscript, noframes or site map if you like, but you definitly need links.

Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved