homepage Welcome to WebmasterWorld Guest from 54.205.106.111
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member
Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt

Sitemaps, Meta Data, and robots.txt

Forum Charter, Library, Moderated by: goodroi Forum Options: Reset Last Read Pointers

Using a robots.txt is all part of being a good SEO. Be sure to check yours in the robots.txt validator that is available to subscribers.

 
new Subject Messages Started by Last Message
newpost thread iconparameter page to be banned from crawling
  directory.asp?
4idolw 1:18 pm on Dec 15, 2005 (gmt 0)
newpost thread iconAbout the robots tag for Googlebot
  Is it necessary?
2a1call 8:57 pm on Dec 14, 2005 (gmt 0)
newpost thread iconRemove part of your website
  Should I use robots.txt or 301 redirects?
1twist 7:27 pm on Dec 13, 2005 (gmt 0)
newpost thread iconRobots.txt and the Google sandbox
  Does de-indexing put one back into the sandbox?
2Rollo 7:05 pm on Dec 13, 2005 (gmt 0)
newpost thread iconShould I use the "?" or not.
 
2kgormat 3:52 am on Dec 12, 2005 (gmt 0)
newpost thread iconrobots.txt and the law
  do they intersect
2Powdork 10:19 am on Dec 11, 2005 (gmt 0)
newpost thread iconNewbie here require help
  robots.txt
2sim20 11:36 pm on Dec 2, 2005 (gmt 0)
newpost thread icondisallow URL with parameters
  /?id=asdf vs /
5Ratbert 11:18 am on Dec 1, 2005 (gmt 0)
newpost thread iconBan Google from 1 directory
  Ban Google from 1 directory
2dunhill 6:47 am on Nov 30, 2005 (gmt 0)
newpost thread iconYahoo Ignoring robots.txt
  Anyone know how to get Yahoo....
2marketingmagic 8:23 pm on Nov 29, 2005 (gmt 0)
newpost thread iconnoimageindex?
  How do I allow a page to be cached without right clicking allowed?
3NeedALife 7:44 pm on Nov 29, 2005 (gmt 0)
newpost thread iconRobots.txt not functioning!
  Added the file months ago but results still coming up
4nwhorton 6:55 pm on Nov 28, 2005 (gmt 0)
newpost thread iconlist of robots to disallow that actually obey robots.txt
 
1koen 3:32 pm on Nov 28, 2005 (gmt 0)
newpost thread icondoes robots.txt really effect ranking?
 
2goneinthesun 4:44 pm on Nov 26, 2005 (gmt 0)
newpost thread iconForce spider to revisit sooner?
  Robots instructions help.
4Fuzzi_Bear 2:32 pm on Nov 26, 2005 (gmt 0)
newpost thread iconGeting lots of hits on robots.txt but not much indexes in Yahoo/Google
 
8ball4121 4:47 am on Nov 26, 2005 (gmt 0)
newpost thread iconsimple robots.txt question
  just to make sure
5followgreg 4:24 am on Nov 26, 2005 (gmt 0)
newpost thread iconRobots.txt Vs META name="robots"
  Which is better?
4Fuzzi_Bear 8:46 pm on Nov 25, 2005 (gmt 0)
newpost thread iconQuick robots.txt help
  is this the right syntax?
3Fuzzi_Bear 2:17 pm on Nov 25, 2005 (gmt 0)
newpost thread iconban all robots for a short period
 
11digicam 6:02 pm on Nov 23, 2005 (gmt 0)
newpost thread iconWebmasterWorld robots.txt
 
5phpmaven 4:45 pm on Nov 23, 2005 (gmt 0)
newpost thread iconMobile content
  Mobile content might be seen as Duplicate.
1designasp 12:41 am on Nov 23, 2005 (gmt 0)
newpost thread iconGap - how many days
 
4successlieswithme 9:39 am on Nov 22, 2005 (gmt 0)
newpost thread iconwhy has google not removed these url's
 
3fzx5v0 3:48 am on Nov 21, 2005 (gmt 0)
newpost thread iconCapitalize robots.txt filename?
  Robots.txt or robots.txt
34string 4:18 am on Nov 17, 2005 (gmt 0)
Displaying Page 41 of 71<< Back 1 ... 31 32 33 34 35 36 37 38 39 40 [ 41 ] 42 43 44 45 46 47 48 49 50 ... 71 Next >>
Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved