homepage Welcome to WebmasterWorld Guest from 54.166.173.147
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Pubcon Platinum Sponsor 2014
Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt

Sitemaps, Meta Data, and robots.txt

Forum Charter, Library, Moderated by: goodroi Forum Options: Reset Last Read Pointers

Using a robots.txt is all part of being a good SEO. Be sure to check yours in the robots.txt validator that is available to subscribers.

 
new Subject Messages Started by Last Message
newpost thread iconGoogle 5000 character limit for robots.txt?
 
6physics 8:53 pm on Jul 1, 2006 (gmt 0)
newpost thread iconhow to block all robots except for 2?
  possible?
7ionchannels 12:30 pm on Jun 29, 2006 (gmt 0)
newpost thread iconRe-Indexing Timetable
  Any guesses as to when the SE's will display our listings again.
4kgormat 7:42 pm on Jun 24, 2006 (gmt 0)
newpost thread iconDisallow: /#example.html#
  How would the following be viewed by a spider?
4kgormat 3:29 pm on Jun 24, 2006 (gmt 0)
newpost thread iconRobot behaviour
  using the robots meta correctly
1Skylo 12:47 pm on Jun 20, 2006 (gmt 0)
newpost thread iconUnderstanding Robot Txt Validator Results
  I don't know what my results mean.
4Caran 4:28 pm on Jun 17, 2006 (gmt 0)
newpost thread iconIs the robots.txt header imp?
  It is passing html header to plain text
5AjiNIMC 6:15 am on Jun 15, 2006 (gmt 0)
newpost thread iconrobots.txt format for different SE
  need help to findout correct format
2token128 12:38 am on Jun 10, 2006 (gmt 0)
newpost thread iconsimple robots.txt clarification
  folders within folders
5headown 8:55 pm on Jun 9, 2006 (gmt 0)
newpost thread iconRecently added robots.txt, validated, but pages still showing up
  Validated in Google Sitemaps but pages still indexed
2blueheelers 8:45 am on Jun 6, 2006 (gmt 0)
newpost thread iconAlexa revealing too much information once again
  Alexa revealing too much information once again
2Gomvents 12:18 pm on Jun 5, 2006 (gmt 0)
newpost thread iconwildcard
 
8stevelibby 12:53 am on May 31, 2006 (gmt 0)
newpost thread iconAdsBot-Google's robots.txt specs
  (FYI from the "Google AdWords" Forum)
1Pfui 12:48 am on May 28, 2006 (gmt 0)
newpost thread iconcan someone please check this.
  can someone confirm this robots.txt is correct...
3keentolearn 9:51 am on May 27, 2006 (gmt 0)
newpost thread iconSub Domain Issue
 
2is300 4:05 pm on May 25, 2006 (gmt 0)
newpost thread iconCache renders excluded CSS?
 
8glengara 10:16 am on May 24, 2006 (gmt 0)
newpost thread iconHaving 2 versions of a site, how to not get one spidered
  Non SE friendly and SEO site
1epadgett 4:28 pm on May 23, 2006 (gmt 0)
newpost thread iconRobots.txt
  Validation - Robots.txt
3Suman 1:09 am on May 23, 2006 (gmt 0)
newpost thread iconrobots.txt and https
 
2AzDude 8:56 pm on May 18, 2006 (gmt 0)
newpost thread iconyahoo slurping for www.file1.html/robots.txt
  what's wrong that slurp can't figure out?
3paybacksa 9:08 am on May 14, 2006 (gmt 0)
newpost thread iconDo search Engines crawl and spider registered content?
  Spiders, crawlers, indexing, registered content
6Oimachi2 9:17 am on May 13, 2006 (gmt 0)
newpost thread iconHow do i add a robots.txt.?
  New to all this and havent got a clue
3gooders2274 1:00 pm on May 12, 2006 (gmt 0)
newpost thread iconCant see competions robots.txt
 
4da95649 12:56 pm on May 12, 2006 (gmt 0)
newpost thread iconoodle crawler
  how to block it?
2moneybagsuk 12:52 am on May 10, 2006 (gmt 0)
newpost thread iconRobots and SMF PHP Forum Script
 
1freewebsiteideas 11:20 pm on May 9, 2006 (gmt 0)
Displaying Page 36 of 71<< Back 1 ... 26 27 28 29 30 31 32 33 34 35 [ 36 ] 37 38 39 40 41 42 43 44 45 ... 71 Next >>
Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved