homepage Welcome to WebmasterWorld Guest from 54.224.202.109
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member
Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt

Sitemaps, Meta Data, and robots.txt

Forum Charter, Library, Moderated by: goodroi Forum Options: Reset Last Read Pointers

Using a robots.txt is all part of being a good SEO. Be sure to check yours in the robots.txt validator that is available to subscribers.

 
new Subject Messages Started by Last Message
newpost thread iconRe-Indexing Timetable
  Any guesses as to when the SE's will display our listings again.
4kgormat 7:42 pm on Jun 24, 2006 (gmt 0)
newpost thread iconDisallow: /#example.html#
  How would the following be viewed by a spider?
4kgormat 3:29 pm on Jun 24, 2006 (gmt 0)
newpost thread iconRobot behaviour
  using the robots meta correctly
1Skylo 12:47 pm on Jun 20, 2006 (gmt 0)
newpost thread iconUnderstanding Robot Txt Validator Results
  I don't know what my results mean.
4Caran 4:28 pm on Jun 17, 2006 (gmt 0)
newpost thread iconIs the robots.txt header imp?
  It is passing html header to plain text
5AjiNIMC 6:15 am on Jun 15, 2006 (gmt 0)
newpost thread iconrobots.txt format for different SE
  need help to findout correct format
2token128 12:38 am on Jun 10, 2006 (gmt 0)
newpost thread iconsimple robots.txt clarification
  folders within folders
5headown 8:55 pm on Jun 9, 2006 (gmt 0)
newpost thread iconRecently added robots.txt, validated, but pages still showing up
  Validated in Google Sitemaps but pages still indexed
2blueheelers 8:45 am on Jun 6, 2006 (gmt 0)
newpost thread iconAlexa revealing too much information once again
  Alexa revealing too much information once again
2Gomvents 12:18 pm on Jun 5, 2006 (gmt 0)
newpost thread iconwildcard
 
8stevelibby 12:53 am on May 31, 2006 (gmt 0)
newpost thread iconAdsBot-Google's robots.txt specs
  (FYI from the "Google AdWords" Forum)
1Pfui 12:48 am on May 28, 2006 (gmt 0)
newpost thread iconcan someone please check this.
  can someone confirm this robots.txt is correct...
3keentolearn 9:51 am on May 27, 2006 (gmt 0)
newpost thread iconSub Domain Issue
 
2is300 4:05 pm on May 25, 2006 (gmt 0)
newpost thread iconCache renders excluded CSS?
 
8glengara 10:16 am on May 24, 2006 (gmt 0)
newpost thread iconHaving 2 versions of a site, how to not get one spidered
  Non SE friendly and SEO site
1epadgett 4:28 pm on May 23, 2006 (gmt 0)
newpost thread iconRobots.txt
  Validation - Robots.txt
3Suman 1:09 am on May 23, 2006 (gmt 0)
newpost thread iconrobots.txt and https
 
2AzDude 8:56 pm on May 18, 2006 (gmt 0)
newpost thread iconyahoo slurping for www.file1.html/robots.txt
  what's wrong that slurp can't figure out?
3paybacksa 9:08 am on May 14, 2006 (gmt 0)
newpost thread iconDo search Engines crawl and spider registered content?
  Spiders, crawlers, indexing, registered content
6Oimachi2 9:17 am on May 13, 2006 (gmt 0)
newpost thread iconHow do i add a robots.txt.?
  New to all this and havent got a clue
3gooders2274 1:00 pm on May 12, 2006 (gmt 0)
newpost thread iconCant see competions robots.txt
 
4da95649 12:56 pm on May 12, 2006 (gmt 0)
newpost thread iconoodle crawler
  how to block it?
2moneybagsuk 12:52 am on May 10, 2006 (gmt 0)
newpost thread iconRobots and SMF PHP Forum Script
 
1freewebsiteideas 11:20 pm on May 9, 2006 (gmt 0)
newpost thread iconSubdomains, DNS, IIS
  How to handle robots.txt requests for multiple subdomains...
1jtoddv 9:23 pm on May 8, 2006 (gmt 0)
newpost thread iconShould I use this template?
  Very silly question of a beginner...
4wolfadeus 9:53 am on May 4, 2006 (gmt 0)
Displaying Page 36 of 71<< Back 1 ... 26 27 28 29 30 31 32 33 34 35 [ 36 ] 37 38 39 40 41 42 43 44 45 ... 71 Next >>
Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved