homepage Welcome to WebmasterWorld Guest from 23.21.9.44
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Pubcon Platinum Sponsor 2014
Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt

Sitemaps, Meta Data, and robots.txt

Forum Charter, Library, Moderated by: goodroi Forum Options: Reset Last Read Pointers

Using a robots.txt is all part of being a good SEO. Be sure to check yours in the robots.txt validator that is available to subscribers.

 
new Subject Messages Started by Last Message
newpost thread iconrobots.txt elementary question
 
3Vespasian 5:29 am on Mar 24, 2004 (gmt 0)
newpost thread iconHow to identify search engine robots?
  What's the best way for ASP developers to identify robots?
1vieth 3:48 pm on Mar 23, 2004 (gmt 0)
newpost thread iconRobot text for dissallow asp pages indexed
 
2DivinoStore 8:06 am on Mar 21, 2004 (gmt 0)
newpost thread iconAnyone recognize this agent?
  Can't find any info about it anywhere
2martyt 4:15 pm on Mar 20, 2004 (gmt 0)
newpost thread iconA new bot?
  Anyone else seeing this?
4walrus 6:48 am on Mar 19, 2004 (gmt 0)
newpost thread iconCan't prevent php files from being searched
  Is this possible
1Jack_mcs 1:58 pm on Mar 18, 2004 (gmt 0)
newpost thread iconnewbie robots.txt and forum Q
  how to write it
6buksida 3:27 am on Mar 18, 2004 (gmt 0)
newpost thread iconRobot.txt File
  What is it's importance?
7Enigmatic 8:52 pm on Mar 16, 2004 (gmt 0)
newpost thread iconQuepasaCreep Agent
  where is it from?
3barbeta 8:17 pm on Mar 16, 2004 (gmt 0)
newpost thread iconDoes robots exclusion prevent pr being passed
 
2djtaverner 5:38 pm on Mar 16, 2004 (gmt 0)
newpost thread iconShould I ban this user agent?
 
4jehoshua 5:13 pm on Mar 16, 2004 (gmt 0)
newpost thread iconsyntax for all php exclusion
 
2djtaverner 4:29 pm on Mar 15, 2004 (gmt 0)
newpost thread iconCannot find error
  Robots.txt syntax error
5webapache 5:04 pm on Mar 14, 2004 (gmt 0)
newpost thread iconrobots.txt and mod_rewrite
  just double checking
6hannamyluv 2:24 am on Mar 14, 2004 (gmt 0)
newpost thread iconSeveral robots.txt?
 
2brakkar 1:18 am on Mar 14, 2004 (gmt 0)
newpost thread iconWhere is the "Top Level"?
  is this important these days?
4chewy 9:46 pm on Mar 13, 2004 (gmt 0)
newpost thread iconDoes this work?
  Disallow: /somepath/*/file/*
6DoppyNL 5:36 pm on Mar 13, 2004 (gmt 0)
newpost thread iconCCGCrawl -- custom crawler/harvester software
  New crawler software potential for abuse
2privacyman 1:38 pm on Mar 13, 2004 (gmt 0)
newpost thread iconHow to exclude bots from specific domains if all point to same place?
  We have foreign domains that all point to the same folder
2alpine 6:06 am on Mar 13, 2004 (gmt 0)
newpost thread iconYahoo Images
 
2RammsteinNicCage 5:56 am on Mar 13, 2004 (gmt 0)
newpost thread iconWho is NaverBot-1.0 (NHN Corp.
  Just wondering what bot this is....
2RobbieD 11:51 am on Mar 12, 2004 (gmt 0)
newpost thread iconmeta-robots tag; wil link be followed or not?
 
2DoppyNL 5:30 am on Mar 12, 2004 (gmt 0)
newpost thread iconrobots.txt not working
  Disallows are still being indexed
3jchance 5:21 am on Mar 12, 2004 (gmt 0)
newpost thread iconCan't seem to block Xenu
 
6WebBender 12:55 am on Mar 12, 2004 (gmt 0)
newpost thread iconphp and 'include' files
 
3charlie12345 3:19 am on Mar 11, 2004 (gmt 0)
Displaying Page 59 of 71<< Back 1 ... 49 50 51 52 53 54 55 56 57 58 [ 59 ] 60 61 62 63 64 65 66 67 68 ... 71 Next >>
Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved