homepage Welcome to WebmasterWorld Guest from 54.145.183.126
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member
Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt

Sitemaps, Meta Data, and robots.txt

Forum Charter, Library, Moderated by: goodroi Forum Options: Reset Last Read Pointers

Using a robots.txt is all part of being a good SEO. Be sure to check yours in the robots.txt validator that is available to subscribers.

 
new Subject Messages Started by Last Message
newpost thread icongooglebot, robots.txt and spidering
  Perplexed by lack or spidering of the site
8partnermine 5:12 am on May 6, 2005 (gmt 0)
newpost thread iconmeta tags+robots.txt
  newbie question
3bonkph 8:16 pm on May 5, 2005 (gmt 0)
newpost thread iconBaiduspider - not checking robots.txt & how to block?
 
6TreeShare 8:07 am on May 5, 2005 (gmt 0)
newpost thread iconrobots.txt and subdomain
  Can I use this file to block a specific subdomain
4Pat1975 8:57 am on May 4, 2005 (gmt 0)
newpost thread icongoogle hits robots.txt but does not index anything?
  am i doing something wrong?
19mbatta 5:38 am on May 4, 2005 (gmt 0)
newpost thread iconCan someone tell me how to ........
 
3Lex_Luther 9:00 pm on May 3, 2005 (gmt 0)
newpost thread iconGoogle - User Agent: * Disallow: /
 
4wchan07 4:42 pm on May 3, 2005 (gmt 0)
newpost thread iconmod_rewrite and robots.txt
 
2jewels 3:03 pm on May 3, 2005 (gmt 0)
newpost thread iconBlocking Yahoo from a specific folder
  How do I do this?
2Rollo 2:55 pm on May 3, 2005 (gmt 0)
newpost thread iconHow to Validate robots.txt and do I even have one?
 
13neophitemoneymaker 11:58 pm on May 1, 2005 (gmt 0)
newpost thread iconPage indexed in Google despite robots.txt entry
  How did they find it in the first place?
23pmkpmk 9:17 pm on May 1, 2005 (gmt 0)
newpost thread iconThe Great SearchEngineWorld robots.txt Survey #2 is Under Way
 
9Brett_Tabke 2:35 pm on Apr 29, 2005 (gmt 0)
newpost thread iconNo-Follow, allow indexing of a file but not it's links?
 
2JAB_Creations 8:32 am on Apr 27, 2005 (gmt 0)
newpost thread iconHow to stop Yahoo Slurp?
  Slurp ingnores my robots.txt
16atlrus 9:47 pm on Apr 26, 2005 (gmt 0)
newpost thread iconHow to remove ezboard forum from Wayback?
 
3wmw99 12:04 pm on Apr 26, 2005 (gmt 0)
newpost thread iconIs there a definitive robots.txt to disallow bad bots
  bad bots and bad bots only
6Clark 11:40 am on Apr 24, 2005 (gmt 0)
newpost thread iconDisallow keyword Ignored?
  Who guarantees me that disallow keyword isn't ignored
5bkausbk 9:02 am on Apr 24, 2005 (gmt 0)
newpost thread iconAllowing and disallowing spiders
  Easy one - can somebody confirm my understanding?
3Zeberdy 9:19 am on Apr 21, 2005 (gmt 0)
newpost thread iconHow to disallow getlastpost
 
2annelisabeth 9:19 am on Apr 21, 2005 (gmt 0)
newpost thread iconRobots.txt Help request.
  I need help with stopping spyder robots.
10westmore 6:22 am on Apr 21, 2005 (gmt 0)
newpost thread iconMeta Descriptions
  How many can I have?
8Andromeda 8:07 pm on Apr 20, 2005 (gmt 0)
newpost thread iconRobots.txt Review
  I need a confirmation for robots.txt from you experts
11RaniaD 11:51 am on Apr 20, 2005 (gmt 0)
newpost thread iconI want to disallow extra for Slurp
  is this syntax right?
2baze22 12:55 pm on Apr 19, 2005 (gmt 0)
newpost thread iconRobots.txt vs Robots META tag
  Does one over-rule the other?
3obsos 7:14 am on Apr 18, 2005 (gmt 0)
newpost thread iconValidation of robots.txt
 
3designei 10:50 am on Apr 17, 2005 (gmt 0)
Displaying Page 48 of 72<< Back 1 ... 38 39 40 41 42 43 44 45 46 47 [ 48 ] 49 50 51 52 53 54 55 56 57 ... 72 Next >>
Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved