Welcome to WebmasterWorld Guest from 126.96.36.199
Using a robots.txt is all part of being a good SEO. Be sure to check yours in the robots.txt validator that is available to subscribers.
|Thread Subject||Messages||Started by||Last Message|
||2||stormshield||5:13 pm Jul 6, 2008|
|Dynamic URL's and robots.txt
||4||cgydosh||3:37 pm Jul 2, 2008|
|How to setup robots.txt?
||3||Cameron08||10:23 am Jun 27, 2008|
|Indexing problem for few pages in Google
I have optimized one static web site vinsystech.com.
|3||venuakkala||12:49 pm Jun 24, 2008|
|Syntax for disallowing folder with '.' in the name?
Asking for clarification
|3||ALbino||10:40 pm Jun 23, 2008|
|Google ignores one line of robots.txt
||8||wheelie34||9:16 am Jun 22, 2008|
|Google under cover?
||3||Guido||1:38 pm Jun 13, 2008|
|Need to block and remove 1 page from serps
||4||StaceyJ||8:32 pm Jun 6, 2008|
||3||hugh||12:49 pm Jun 6, 2008|
|Home page blocked accidentally by robots tag
home page dropped from serps
|5||jeffe||6:19 pm May 21, 2008|
|robots.txt file being ignored and old content not being removed
||10||Marty3454||11:46 pm May 20, 2008|
|To use or not to use a robots.txt file
||3||con771||6:03 pm May 20, 2008|
|4 Wordpress File's Link To Be Blocked
How should I block them?
|5||aced84||3:54 pm May 19, 2008|
|I think there is a problem
|2||AllanS||10:39 am May 17, 2008|
|robots.txt syntax for wildcards?
||3||Marfola||7:15 am May 15, 2008|
|How many links can i place in one page of site map?
||5||bava_seo||2:12 pm May 10, 2008|
|why use and what is robots.txt
this maybe stupid question
|3||mick_simon||4:45 pm May 7, 2008|
|How To stop crawler to crawl https:// pages
||8||vreelmistee1||6:03 am May 7, 2008|
|Status: 404(not found) in Google Webmaster Tools
||3||Boulder90||6:39 pm May 6, 2008|
|Y! and robots.txt
|5||Staffa||6:36 pm May 6, 2008|
|Googlebot is Not Taking Visit to my site
robots.txt is restricted googlebot to do not visit
|4||bilalseo||1:54 pm May 6, 2008|
|Site Pages are Blocked by Robots.txt
||4||bilalseo||10:03 am May 6, 2008|
|robots.txt and IP addresses
||11||Maciej_Ziemczonek||10:11 pm May 2, 2008|
|Is this valid? wildcard in robots
|3||rudyten||1:47 pm Apr 20, 2008|
|Will spiders follow links on URL's blocked by robots.txt
||3||deanw||6:34 pm Apr 16, 2008|