homepage Welcome to WebmasterWorld Guest from 54.204.64.152
register, free tools, login, search, subscribe, help, library, announcements, recent posts, open posts,
Subscribe and Support WebmasterWorld
Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt

Sitemaps, Meta Data, and robots.txt

Forum Charter, Library, Moderated by: goodroi Forum Options: Reset Last Read Pointers

Using a robots.txt is all part of being a good SEO. Be sure to check yours in the robots.txt validator that is available to subscribers.

 
new Subject Messages Started by Last Message
newpost thread iconrobots.txt to stop some domains
  How to allow only one domain to be indexed for the same website.
5john_pinx 3:39 am on Oct 12, 2004 (gmt 0)
newpost thread iconduhhhhh, dumb blonde needing help with robot.txt
  I don't have a clue - go easy please....
4ladyluck 6:37 am on Oct 9, 2004 (gmt 0)
newpost thread iconHow to Disallow a Particular Page
 
5jsnively 9:25 pm on Oct 6, 2004 (gmt 0)
newpost thread iconhacking robots.txt for RSS features?
 
1phildog 3:53 pm on Oct 4, 2004 (gmt 0)
newpost thread iconIs robots.txt always needed?
  robots.txt
10benwalsh 12:38 pm on Oct 4, 2004 (gmt 0)
newpost thread iconbad bots
 
4rhodopsin 11:40 am on Oct 3, 2004 (gmt 0)
newpost thread iconInsert disallow.php to robots
  is it a good idea?
3sirkei 5:52 am on Oct 3, 2004 (gmt 0)
newpost thread iconCan i add robot txt to a free hosting site? keywords
 
2alze 5:14 am on Oct 3, 2004 (gmt 0)
newpost thread iconDoes robots.txt prevent all access?
  ...on page as well as directory.
2yump 5:02 am on Oct 3, 2004 (gmt 0)
newpost thread iconRobots.txt sabotage
  Common? How to protect?
4adfree 11:05 am on Sep 30, 2004 (gmt 0)
newpost thread iconnewbie to robots.txt, want to exclude /images/
  excluding some image folders but not others
2franco81 3:29 am on Sep 30, 2004 (gmt 0)
newpost thread iconNewbie - Why wouldn't I want to allow certain robots?
  If I don't have anything on the site I want to keep private
4AmyG 4:03 pm on Sep 29, 2004 (gmt 0)
newpost thread iconWhat if your domains robots.txt
  Was pointed to your domain?
5Daily_Sparring 3:22 am on Sep 27, 2004 (gmt 0)
newpost thread iconWhere to put robots in website?
  I has doubt where to put robots.
3viva 10:00 am on Sep 26, 2004 (gmt 0)
newpost thread iconrobots.txt and .htaccess
 
1HuhuFruFru 12:02 am on Sep 26, 2004 (gmt 0)
newpost thread iconExcluding Sub Folders
  When you exclude a folder, are folders inside that excluded also?
3damlag 8:39 pm on Sep 25, 2004 (gmt 0)
newpost thread iconCan someone help me write a robots.txt?
 
5budbiss 11:50 pm on Sep 19, 2004 (gmt 0)
newpost thread iconwww.example.com/view.php
  How to avoid spidering of anything with this prefix
2nuevojefe 10:26 pm on Sep 19, 2004 (gmt 0)
newpost thread iconrobots.txt
  Invalid warnings
5wolfline 5:10 am on Sep 17, 2004 (gmt 0)
newpost thread iconpossible sabotage?
  robot.txt working against me?
5seofan 9:36 pm on Sep 8, 2004 (gmt 0)
newpost thread iconwho can give me a website log file sample to see?
  website log file
3Tellist 5:19 pm on Sep 4, 2004 (gmt 0)
newpost thread iconGoogle CHTML Proxy disobeying robots.txt
  Why?
5yosmc 12:16 pm on Aug 31, 2004 (gmt 0)
newpost thread iconGoogle tells me to change my robots.txt
  Should I follow the advice from Google adSense?
2Bonusbana 2:44 pm on Aug 27, 2004 (gmt 0)
newpost thread iconMaster List of Malicious Robots
  Is there an active list of the bots we should block?
5Eric_Lander 4:09 pm on Aug 25, 2004 (gmt 0)
newpost thread iconrobots.txt security problems?
 
2helohelo 3:44 pm on Aug 25, 2004 (gmt 0)
Displaying Page 54 of 71<< Back 1 ... 44 45 46 47 48 49 50 51 52 53 [ 54 ] 55 56 57 58 59 60 61 62 63 ... 71 Next >>
Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About
© Webmaster World 1996-2014 all rights reserved