homepage Welcome to WebmasterWorld Guest from 54.205.59.78
register, free tools, login, search, subscribe, help, library, announcements, recent posts, open posts,
Subscribe to WebmasterWorld
Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt

Sitemaps, Meta Data, and robots.txt

Forum Charter, Library, Moderated by: goodroi Forum Options: Reset Last Read Pointers

Using a robots.txt is all part of being a good SEO. Be sure to check yours in the robots.txt validator that is available to subscribers.

 
new Subject Messages Started by Last Message
newpost thread iconOrder allow,deny or deny,allow
  Not sure which order is best...
3whoisgregg 9:16 pm on May 27, 2005 (gmt 0)
newpost thread iconSearch Engine Crawlers
 
2dayo 8:39 pm on May 26, 2005 (gmt 0)
newpost thread iconSpidering of my code pages
  Wrong pages being spidered and losing referals
5djfelip 11:18 pm on May 25, 2005 (gmt 0)
newpost thread iconrobots4.txt -- I'm confused
  Confused about what is allowed in this file
3Goose_68 8:46 pm on May 24, 2005 (gmt 0)
newpost thread iconnewbie question
  http://www.yourdomain.com/robots.txt
2webmz 3:10 am on May 22, 2005 (gmt 0)
newpost thread iconValidation
 
18Dexie 4:14 pm on May 21, 2005 (gmt 0)
newpost thread iconCan't find the robot.txt
  where is it?
27mmmwowmmm 10:56 am on May 21, 2005 (gmt 0)
newpost thread iconSonicWALL Firewall Blocks Spiders
  Settings will block spiders from indexing site.
26EliteWeb 3:16 pm on May 15, 2005 (gmt 0)
newpost thread iconRobots.txt Format
  Will this work?
4kbba04527 6:37 am on May 6, 2005 (gmt 0)
newpost thread icongooglebot, robots.txt and spidering
  Perplexed by lack or spidering of the site
8partnermine 5:12 am on May 6, 2005 (gmt 0)
newpost thread iconmeta tags+robots.txt
  newbie question
3bonkph 8:16 pm on May 5, 2005 (gmt 0)
newpost thread iconBaiduspider - not checking robots.txt & how to block?
 
6TreeShare 8:07 am on May 5, 2005 (gmt 0)
newpost thread iconrobots.txt and subdomain
  Can I use this file to block a specific subdomain
4Pat1975 8:57 am on May 4, 2005 (gmt 0)
newpost thread icongoogle hits robots.txt but does not index anything?
  am i doing something wrong?
19mbatta 5:38 am on May 4, 2005 (gmt 0)
newpost thread iconCan someone tell me how to ........
 
3Lex_Luther 9:00 pm on May 3, 2005 (gmt 0)
newpost thread iconGoogle - User Agent: * Disallow: /
 
4wchan07 4:42 pm on May 3, 2005 (gmt 0)
newpost thread iconmod_rewrite and robots.txt
 
2jewels 3:03 pm on May 3, 2005 (gmt 0)
newpost thread iconBlocking Yahoo from a specific folder
  How do I do this?
2Rollo 2:55 pm on May 3, 2005 (gmt 0)
newpost thread iconHow to Validate robots.txt and do I even have one?
 
13neophitemoneymaker 11:58 pm on May 1, 2005 (gmt 0)
newpost thread iconPage indexed in Google despite robots.txt entry
  How did they find it in the first place?
23pmkpmk 9:17 pm on May 1, 2005 (gmt 0)
newpost thread iconThe Great SearchEngineWorld robots.txt Survey #2 is Under Way
 
9Brett_Tabke 2:35 pm on Apr 29, 2005 (gmt 0)
newpost thread iconNo-Follow, allow indexing of a file but not it's links?
 
2JAB_Creations 8:32 am on Apr 27, 2005 (gmt 0)
newpost thread iconHow to stop Yahoo Slurp?
  Slurp ingnores my robots.txt
16atlrus 9:47 pm on Apr 26, 2005 (gmt 0)
newpost thread iconHow to remove ezboard forum from Wayback?
 
3wmw99 12:04 pm on Apr 26, 2005 (gmt 0)
newpost thread iconIs there a definitive robots.txt to disallow bad bots
  bad bots and bad bots only
6Clark 11:40 am on Apr 24, 2005 (gmt 0)
Displaying Page 47 of 71<< Back 1 ... 37 38 39 40 41 42 43 44 45 46 [ 47 ] 48 49 50 51 52 53 54 55 56 ... 71 Next >>
Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About
© Webmaster World 1996-2014 all rights reserved