homepage Welcome to WebmasterWorld Guest from 54.197.189.108
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member

Visit PubCon.com
Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt

Sitemaps, Meta Data, and robots.txt

Forum Charter, Library, Moderated by: goodroi Forum Options: Reset Last Read Pointers

Using a robots.txt is all part of being a good SEO. Be sure to check yours in the robots.txt validator that is available to subscribers.

 
new Subject Messages Started by Last Message
newpost thread iconIf I disallow /filename.php ...
  will google still index /filename.php?whatever
7Trisha 7:05 am on Jun 7, 2005 (gmt 0)
newpost thread iconAgent Name for MSN
  Where can I find this information.
8Pat1975 5:55 am on Jun 3, 2005 (gmt 0)
newpost thread iconGeneral questions
  General robots.txt queries
7makzan 11:49 pm on Jun 2, 2005 (gmt 0)
newpost thread iconDisallow /file.php same as Diallow file.php?
 
6One_on_One 3:43 am on Jun 2, 2005 (gmt 0)
newpost thread iconStopping spiders
  from Competitors Analysis Tools...
4ambiorix_rb 1:21 pm on Jun 1, 2005 (gmt 0)
newpost thread iconSEO / robots.txt question
  unintentional cross-linking penalty
1wolfgang 6:27 pm on May 31, 2005 (gmt 0)
newpost thread iconBlocking a Sub Domain Directory
 
3is300 8:25 am on May 30, 2005 (gmt 0)
newpost thread iconOrder allow,deny or deny,allow
  Not sure which order is best...
3whoisgregg 9:16 pm on May 27, 2005 (gmt 0)
newpost thread iconSearch Engine Crawlers
 
2dayo 8:39 pm on May 26, 2005 (gmt 0)
newpost thread iconSpidering of my code pages
  Wrong pages being spidered and losing referals
5djfelip 11:18 pm on May 25, 2005 (gmt 0)
newpost thread iconrobots4.txt -- I'm confused
  Confused about what is allowed in this file
3Goose_68 8:46 pm on May 24, 2005 (gmt 0)
newpost thread iconnewbie question
  http://www.yourdomain.com/robots.txt
2webmz 3:10 am on May 22, 2005 (gmt 0)
newpost thread iconValidation
 
18Dexie 4:14 pm on May 21, 2005 (gmt 0)
newpost thread iconCan't find the robot.txt
  where is it?
27mmmwowmmm 10:56 am on May 21, 2005 (gmt 0)
newpost thread iconSonicWALL Firewall Blocks Spiders
  Settings will block spiders from indexing site.
26EliteWeb 3:16 pm on May 15, 2005 (gmt 0)
newpost thread iconRobots.txt Format
  Will this work?
4kbba04527 6:37 am on May 6, 2005 (gmt 0)
newpost thread icongooglebot, robots.txt and spidering
  Perplexed by lack or spidering of the site
8partnermine 5:12 am on May 6, 2005 (gmt 0)
newpost thread iconmeta tags+robots.txt
  newbie question
3bonkph 8:16 pm on May 5, 2005 (gmt 0)
newpost thread iconBaiduspider - not checking robots.txt & how to block?
 
6TreeShare 8:07 am on May 5, 2005 (gmt 0)
newpost thread iconrobots.txt and subdomain
  Can I use this file to block a specific subdomain
4Pat1975 8:57 am on May 4, 2005 (gmt 0)
newpost thread icongoogle hits robots.txt but does not index anything?
  am i doing something wrong?
19mbatta 5:38 am on May 4, 2005 (gmt 0)
newpost thread iconCan someone tell me how to ........
 
3Lex_Luther 9:00 pm on May 3, 2005 (gmt 0)
newpost thread iconGoogle - User Agent: * Disallow: /
 
4wchan07 4:42 pm on May 3, 2005 (gmt 0)
newpost thread iconmod_rewrite and robots.txt
 
2jewels 3:03 pm on May 3, 2005 (gmt 0)
newpost thread iconBlocking Yahoo from a specific folder
  How do I do this?
2Rollo 2:55 pm on May 3, 2005 (gmt 0)
Displaying Page 47 of 71<< Back 1 ... 37 38 39 40 41 42 43 44 45 46 [ 47 ] 48 49 50 51 52 53 54 55 56 ... 71 Next >>
Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved