homepage Welcome to WebmasterWorld Guest from 54.211.182.241
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member
Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt

Sitemaps, Meta Data, and robots.txt

Forum Charter, Library, Moderated by: goodroi Forum Options: Reset Last Read Pointers

Using a robots.txt is all part of being a good SEO. Be sure to check yours in the robots.txt validator that is available to subscribers.

 
new Subject Messages Started by Last Message
newpost thread iconDisallow index.html in Robots.txt
  Is it safe to disallow an index.html file of a website
3pixeladdict 8:48 pm on Aug 4, 2006 (gmt 0)
newpost thread icondisallow every 'filename' except.
  disallow every 'filename' except...
2decbrad 4:12 am on Aug 4, 2006 (gmt 0)
newpost thread iconRobots.txt Exclusion vs. Nofollow Tag
 
3JasperSEO 1:39 am on Aug 1, 2006 (gmt 0)
newpost thread iconKeep robots away from site at least two weeks
 
1zoth 2:43 pm on Jul 29, 2006 (gmt 0)
newpost thread iconRobots.txt exclusion help
  Robots.txt exclusion help
3webguy7 4:57 am on Jul 22, 2006 (gmt 0)
newpost thread iconusing robots.txt
  is it necessary to exclude .php pages
6deborahbaker 11:53 am on Jul 13, 2006 (gmt 0)
newpost thread iconRobots exclusion revisted
  How far have we come with controlling robots?
3pixel_juice 11:12 pm on Jul 7, 2006 (gmt 0)
newpost thread iconNeed to redirect bots from subdomain to main site
 
1nemo1208 3:28 pm on Jul 7, 2006 (gmt 0)
newpost thread iconHelp with Spiders
  Cutting back on the bandwidth
2devnet 4:56 am on Jul 5, 2006 (gmt 0)
newpost thread iconIs there any harm in not having a robots.txt file at all?
 
21Lit 10:55 am on Jul 4, 2006 (gmt 0)
newpost thread icon90% of my traffic is spiders and bots.
  What to do -
4webpro00801 6:41 am on Jul 3, 2006 (gmt 0)
newpost thread icontwo duplicate sites .com .net
  confused on how to handle mirror sites
4jcnh74 9:33 pm on Jul 1, 2006 (gmt 0)
newpost thread iconGoogle 5000 character limit for robots.txt?
 
6physics 8:53 pm on Jul 1, 2006 (gmt 0)
newpost thread iconhow to block all robots except for 2?
  possible?
7ionchannels 12:30 pm on Jun 29, 2006 (gmt 0)
newpost thread iconRe-Indexing Timetable
  Any guesses as to when the SE's will display our listings again.
4kgormat 7:42 pm on Jun 24, 2006 (gmt 0)
newpost thread iconDisallow: /#example.html#
  How would the following be viewed by a spider?
4kgormat 3:29 pm on Jun 24, 2006 (gmt 0)
newpost thread iconRobot behaviour
  using the robots meta correctly
1Skylo 12:47 pm on Jun 20, 2006 (gmt 0)
newpost thread iconUnderstanding Robot Txt Validator Results
  I don't know what my results mean.
4Caran 4:28 pm on Jun 17, 2006 (gmt 0)
newpost thread iconIs the robots.txt header imp?
  It is passing html header to plain text
5AjiNIMC 6:15 am on Jun 15, 2006 (gmt 0)
newpost thread iconrobots.txt format for different SE
  need help to findout correct format
2token128 12:38 am on Jun 10, 2006 (gmt 0)
newpost thread iconsimple robots.txt clarification
  folders within folders
5headown 8:55 pm on Jun 9, 2006 (gmt 0)
newpost thread iconRecently added robots.txt, validated, but pages still showing up
  Validated in Google Sitemaps but pages still indexed
2blueheelers 8:45 am on Jun 6, 2006 (gmt 0)
newpost thread iconAlexa revealing too much information once again
  Alexa revealing too much information once again
2Gomvents 12:18 pm on Jun 5, 2006 (gmt 0)
newpost thread iconwildcard
 
8stevelibby 12:53 am on May 31, 2006 (gmt 0)
newpost thread iconAdsBot-Google's robots.txt specs
  (FYI from the "Google AdWords" Forum)
1Pfui 12:48 am on May 28, 2006 (gmt 0)
Displaying Page 36 of 72<< Back 1 ... 26 27 28 29 30 31 32 33 34 35 [ 36 ] 37 38 39 40 41 42 43 44 45 ... 72 Next >>
Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved