homepage Welcome to WebmasterWorld Guest from 54.197.129.242
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member

Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt

Sitemaps, Meta Data, and robots.txt

Forum Charter, Library, Moderated by: goodroi Forum Options: Reset Last Read Pointers

Using a robots.txt is all part of being a good SEO. Be sure to check yours in the robots.txt validator that is available to subscribers.

 
new Subject Messages Started by Last Message
newpost thread iconHow to Disallow a Particular Page
 
5jsnively 9:25 pm on Oct 6, 2004 (gmt 0)
newpost thread iconhacking robots.txt for RSS features?
 
1phildog 3:53 pm on Oct 4, 2004 (gmt 0)
newpost thread iconIs robots.txt always needed?
  robots.txt
10benwalsh 12:38 pm on Oct 4, 2004 (gmt 0)
newpost thread iconbad bots
 
4rhodopsin 11:40 am on Oct 3, 2004 (gmt 0)
newpost thread iconInsert disallow.php to robots
  is it a good idea?
3sirkei 5:52 am on Oct 3, 2004 (gmt 0)
newpost thread iconCan i add robot txt to a free hosting site? keywords
 
2alze 5:14 am on Oct 3, 2004 (gmt 0)
newpost thread iconDoes robots.txt prevent all access?
  ...on page as well as directory.
2yump 5:02 am on Oct 3, 2004 (gmt 0)
newpost thread iconRobots.txt sabotage
  Common? How to protect?
4adfree 11:05 am on Sep 30, 2004 (gmt 0)
newpost thread iconnewbie to robots.txt, want to exclude /images/
  excluding some image folders but not others
2franco81 3:29 am on Sep 30, 2004 (gmt 0)
newpost thread iconNewbie - Why wouldn't I want to allow certain robots?
  If I don't have anything on the site I want to keep private
4AmyG 4:03 pm on Sep 29, 2004 (gmt 0)
newpost thread iconWhat if your domains robots.txt
  Was pointed to your domain?
5Daily_Sparring 3:22 am on Sep 27, 2004 (gmt 0)
newpost thread iconWhere to put robots in website?
  I has doubt where to put robots.
3viva 10:00 am on Sep 26, 2004 (gmt 0)
newpost thread iconrobots.txt and .htaccess
 
1HuhuFruFru 12:02 am on Sep 26, 2004 (gmt 0)
newpost thread iconExcluding Sub Folders
  When you exclude a folder, are folders inside that excluded also?
3damlag 8:39 pm on Sep 25, 2004 (gmt 0)
newpost thread iconCan someone help me write a robots.txt?
 
5budbiss 11:50 pm on Sep 19, 2004 (gmt 0)
newpost thread iconwww.example.com/view.php
  How to avoid spidering of anything with this prefix
2nuevojefe 10:26 pm on Sep 19, 2004 (gmt 0)
newpost thread iconrobots.txt
  Invalid warnings
5wolfline 5:10 am on Sep 17, 2004 (gmt 0)
newpost thread iconpossible sabotage?
  robot.txt working against me?
5seofan 9:36 pm on Sep 8, 2004 (gmt 0)
newpost thread iconwho can give me a website log file sample to see?
  website log file
3Tellist 5:19 pm on Sep 4, 2004 (gmt 0)
newpost thread iconGoogle CHTML Proxy disobeying robots.txt
  Why?
5yosmc 12:16 pm on Aug 31, 2004 (gmt 0)
newpost thread iconGoogle tells me to change my robots.txt
  Should I follow the advice from Google adSense?
2Bonusbana 2:44 pm on Aug 27, 2004 (gmt 0)
newpost thread iconMaster List of Malicious Robots
  Is there an active list of the bots we should block?
5Eric_Lander 4:09 pm on Aug 25, 2004 (gmt 0)
newpost thread iconrobots.txt security problems?
 
2helohelo 3:44 pm on Aug 25, 2004 (gmt 0)
newpost thread icon301 Redirect domain?
  How to handle robots.txt?
5Mr_Roberto 1:13 am on Aug 24, 2004 (gmt 0)
newpost thread iconCreating robots.txt file.
 
6kabanero 8:34 pm on Aug 18, 2004 (gmt 0)
Displaying Page 55 of 72<< Back 1 ... 45 46 47 48 49 50 51 52 53 54 [ 55 ] 56 57 58 59 60 61 62 63 64 ... 72 Next >>
Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved