homepage Welcome to WebmasterWorld Guest from 54.145.209.80
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Pubcon Platinum Sponsor 2014
Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt

Sitemaps, Meta Data, and robots.txt

Forum Charter, Library, Moderated by: goodroi Forum Options: Reset Last Read Pointers

Using a robots.txt is all part of being a good SEO. Be sure to check yours in the robots.txt validator that is available to subscribers.

 
new Subject Messages Started by Last Message
newpost thread iconRobots.txt sabotage
  Common? How to protect?
4adfree 11:05 am on Sep 30, 2004 (gmt 0)
newpost thread iconnewbie to robots.txt, want to exclude /images/
  excluding some image folders but not others
2franco81 3:29 am on Sep 30, 2004 (gmt 0)
newpost thread iconNewbie - Why wouldn't I want to allow certain robots?
  If I don't have anything on the site I want to keep private
4AmyG 4:03 pm on Sep 29, 2004 (gmt 0)
newpost thread iconWhat if your domains robots.txt
  Was pointed to your domain?
5Daily_Sparring 3:22 am on Sep 27, 2004 (gmt 0)
newpost thread iconWhere to put robots in website?
  I has doubt where to put robots.
3viva 10:00 am on Sep 26, 2004 (gmt 0)
newpost thread iconrobots.txt and .htaccess
 
1HuhuFruFru 12:02 am on Sep 26, 2004 (gmt 0)
newpost thread iconExcluding Sub Folders
  When you exclude a folder, are folders inside that excluded also?
3damlag 8:39 pm on Sep 25, 2004 (gmt 0)
newpost thread iconCan someone help me write a robots.txt?
 
5budbiss 11:50 pm on Sep 19, 2004 (gmt 0)
newpost thread iconwww.example.com/view.php
  How to avoid spidering of anything with this prefix
2nuevojefe 10:26 pm on Sep 19, 2004 (gmt 0)
newpost thread iconrobots.txt
  Invalid warnings
5wolfline 5:10 am on Sep 17, 2004 (gmt 0)
newpost thread iconpossible sabotage?
  robot.txt working against me?
5seofan 9:36 pm on Sep 8, 2004 (gmt 0)
newpost thread iconwho can give me a website log file sample to see?
  website log file
3Tellist 5:19 pm on Sep 4, 2004 (gmt 0)
newpost thread iconGoogle CHTML Proxy disobeying robots.txt
  Why?
5yosmc 12:16 pm on Aug 31, 2004 (gmt 0)
newpost thread iconGoogle tells me to change my robots.txt
  Should I follow the advice from Google adSense?
2Bonusbana 2:44 pm on Aug 27, 2004 (gmt 0)
newpost thread iconMaster List of Malicious Robots
  Is there an active list of the bots we should block?
5Eric_Lander 4:09 pm on Aug 25, 2004 (gmt 0)
newpost thread iconrobots.txt security problems?
 
2helohelo 3:44 pm on Aug 25, 2004 (gmt 0)
newpost thread icon301 Redirect domain?
  How to handle robots.txt?
5Mr_Roberto 1:13 am on Aug 24, 2004 (gmt 0)
newpost thread iconCreating robots.txt file.
 
6kabanero 8:34 pm on Aug 18, 2004 (gmt 0)
newpost thread iconBlocking just one parameter via robots.txt
  Is this even possible?
2werty 7:31 pm on Aug 18, 2004 (gmt 0)
newpost thread iconProblem with spiderbots scrounging for robot.txt
 
3Mr_N 6:49 am on Aug 10, 2004 (gmt 0)
newpost thread iconi have a subdomain I dont want indexed
  is robots.txt the answer?
4chrisandsarah 1:21 pm on Aug 8, 2004 (gmt 0)
newpost thread iconrobots.txt appearing in my log files
  But I have n't got one there!
5uk_webber 2:59 pm on Aug 5, 2004 (gmt 0)
newpost thread iconTo Robots.txt or not
  Do we use both?
4obsos 11:20 pm on Aug 1, 2004 (gmt 0)
newpost thread iconmsn search violating robots.txt!
  `disallowed` images appear in preview on result page
26DoppyNL 4:15 pm on Aug 1, 2004 (gmt 0)
newpost thread iconRobots text file or noindex,nofollow
  Whats better?
4mthorpe 4:18 am on Jul 31, 2004 (gmt 0)
Displaying Page 55 of 72<< Back 1 ... 45 46 47 48 49 50 51 52 53 54 [ 55 ] 56 57 58 59 60 61 62 63 64 ... 72 Next >>
Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved