homepage Welcome to WebmasterWorld Guest from 54.237.184.242
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member
Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt

Sitemaps, Meta Data, and robots.txt

Forum Charter, Library, Moderated by: goodroi Forum Options: Reset Last Read Pointers

Using a robots.txt is all part of being a good SEO. Be sure to check yours in the robots.txt validator that is available to subscribers.

 
new Subject Messages Started by Last Message
newpost thread iconmeta-robots tag; wil link be followed or not?
 
2DoppyNL 5:30 am on Mar 12, 2004 (gmt 0)
newpost thread iconrobots.txt not working
  Disallows are still being indexed
3jchance 5:21 am on Mar 12, 2004 (gmt 0)
newpost thread iconCan't seem to block Xenu
 
6WebBender 12:55 am on Mar 12, 2004 (gmt 0)
newpost thread iconphp and 'include' files
 
3charlie12345 3:19 am on Mar 11, 2004 (gmt 0)
newpost thread iconWeird robot... Google?
 
2motorhaven 2:14 am on Mar 11, 2004 (gmt 0)
newpost thread iconnewbie question
 
2maxidrom11 2:10 am on Mar 11, 2004 (gmt 0)
newpost thread iconHow To Tell Robots to Index My Site Everyday
  I need it to come everyday and update my website content everday
2dhaliwal 9:01 am on Mar 9, 2004 (gmt 0)
newpost thread iconSome robot keeps calling a 301 page
  about 10000 entries till I banned the IP
1walkman 9:25 pm on Mar 8, 2004 (gmt 0)
newpost thread iconHow to Disallow Jeeves?
 
5skuba 7:03 pm on Mar 5, 2004 (gmt 0)
newpost thread iconIs this done correctly
 
2mikemcs 3:00 pm on Mar 3, 2004 (gmt 0)
newpost thread iconNo Robots.txt page
  what's the harm?
7dauction 4:11 am on Mar 2, 2004 (gmt 0)
newpost thread iconRobots.txt files
  Who to Disallow
6Uniterra 5:08 pm on Mar 1, 2004 (gmt 0)
newpost thread iconHow to disallow doorway pages from indexing?
 
4Voyteck 2:30 pm on Mar 1, 2004 (gmt 0)
newpost thread iconDirectory patterns in query strings
  Will robots.txt block them?
2Wail 8:13 pm on Feb 29, 2004 (gmt 0)
newpost thread iconlist of bots to ban
  it's getting ridiculous but don't want to ban any useful ones
2walkman 4:48 pm on Feb 29, 2004 (gmt 0)
newpost thread iconshould the file .htaccess visible
 
7newbies 3:44 am on Feb 28, 2004 (gmt 0)
newpost thread iconCan I disallow an entire directory but one file?
  too many files to list, plus I'd be advertising them
6walkman 4:13 am on Feb 26, 2004 (gmt 0)
newpost thread iconRobots.txt and Index,nofollow
  do they have to match
4graywolf 10:38 pm on Feb 22, 2004 (gmt 0)
newpost thread iconNew to Robots
  New to Robots
7gotmallz 8:32 pm on Feb 22, 2004 (gmt 0)
newpost thread iconhmmm confused
  this makes no sense
1gotmallz 6:15 am on Feb 22, 2004 (gmt 0)
newpost thread iconConfirm I've done this right please
 
9Essel 5:11 pm on Feb 19, 2004 (gmt 0)
newpost thread iconBanning from the entire webserver?
 
2Essel 5:57 pm on Feb 18, 2004 (gmt 0)
newpost thread iconRobots.txt Error
 
2Takshaka 5:24 pm on Feb 18, 2004 (gmt 0)
newpost thread iconRobots.txt and SERPS
  Blocking serps with robots.txt files and more
2MasterKosmos 6:36 pm on Feb 17, 2004 (gmt 0)
newpost thread iconInktomi Spider Behavior
  what does this mean
5graywolf 12:31 am on Feb 17, 2004 (gmt 0)
Displaying Page 60 of 72<< Back 1 ... 50 51 52 53 54 55 56 57 58 59 [ 60 ] 61 62 63 64 65 66 67 68 69 ... 72 Next >>
Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved