Welcome to WebmasterWorld Guest from 54.196.238.210

Forum Moderated by: goodroi

Sitemaps, Meta Data, and robots.txt


Using a robots.txt is all part of being a good SEO. Be sure to check yours in the robots.txt validator that is available to subscribers.

 
Thread SubjectMessagesStarted byLast Message
WebmasterWorld robots.txt
5 phpmaven 4:45 pm Nov 23, 2005
Mobile content
Mobile content might be seen as Duplicate.
1 designasp 12:41 am Nov 23, 2005
Gap - how many days
4 successlieswithme 9:39 am Nov 22, 2005
why has google not removed these url's
3 fzx5v0 3:48 am Nov 21, 2005
Capitalize robots.txt filename?
Robots.txt or robots.txt
3 4string 4:18 am Nov 17, 2005
Need help removing multiple pages
3 tintin74 11:00 am Nov 15, 2005
Help with Disallow
Not sure if I got this right...
3 BillyS 8:52 pm Nov 14, 2005
stop Yahoo's crawler but for specific domains
how to stop yahoo's crawler for specif domains
2 nrueda 11:40 pm Nov 8, 2005
wildcard for googlebot
4 fzx5v0 4:02 pm Nov 7, 2005
Proposing updates to the robots.txt standard
It is time for a change
16 volatilegx 9:20 pm Nov 4, 2005
what i did wrong?
5 fluxer 3:45 pm Nov 4, 2005
protected ffiles and robots
2 StickeR 7:48 pm Nov 3, 2005
Exculde php url with vars using robots.txt
Exclude urls with vars using robots.txt
2 alexmart 7:44 pm Nov 3, 2005
Is Temporary Disallow a Problem Later?
6 bumpaw 2:10 pm Nov 3, 2005
disallow problem
7 Autoweb 8:25 am Nov 3, 2005
Added robots.txt - 3 days later rankings gone!
11 Block19Row13 3:45 pm Nov 2, 2005
OK now I am baffled - question from new to SEO
4 Iwrite 12:38 pm Nov 2, 2005
Robots.text question, disallowing files
10 mikeD 1:37 am Nov 2, 2005
crawl delay in google, msn, yahoo etc.
crawl delay
4 freegk 7:03 pm Nov 1, 2005
Is robots.txt That necessary?
5 makzan 7:02 pm Nov 1, 2005
after google indexs is it too late to use Disallow
2 watercrazed 8:51 pm Oct 30, 2005
Exclude by filename, not directory?
19 joergnw10 7:36 pm Oct 30, 2005
Robots.txt File to Block from Dynamic URLs
Trying to eliminate duplicate content...
3 BillyS 2:07 am Oct 28, 2005
disallowing a folder with its own domain name
subdomain being spidered although it has its own domain name
2 chandrika 8:52 pm Oct 26, 2005
robots.txt in a folder not root
will this work?
3 fatpeter 6:27 am Oct 26, 2005

Featured Threads

Hot Threads This Week

Hot Threads This Month