Welcome to WebmasterWorld Guest from 54.158.51.150

Forum Moderated by: goodroi

Sitemaps, Meta Data, and robots.txt


Using a robots.txt is all part of being a good SEO. Be sure to check yours in the robots.txt validator that is available to subscribers.

 
Thread SubjectMessagesStarted byLast Message
Using allow to index search-only pages
3 timmbbo 12:31 pm Sep 18, 2007
Disallow - wait a day - add pages
1 brevetoxin 1:36 am Sep 12, 2007
Robots.txt to improve ranks
Will blocking of internal pages through robots.txt improve SE rankings
3 pro_seo 5:18 am Sep 10, 2007
Robots INclusions standard?
13 mikomido 12:28 am Sep 9, 2007
blocking the hostsite but not the hosted sites
3 messemaeker 12:12 pm Sep 7, 2007
Blocking Windows Live robot/spider.
2 Tidal2 10:56 am Sep 7, 2007
Do we have to use robots.txt on every website?
for robots.txt issues
5 masood82 8:42 am Sep 6, 2007
How to block tags?
hundreds of undesired links !
3 rashe18 4:33 pm Sep 4, 2007
Blocked through robots.txt . will the page be crawled ever
5 getxb 6:02 am Aug 31, 2007
URI Only Listing
Need a refresher course on this...
9 pageoneresults 12:20 pm Aug 24, 2007
robot.txt Security
Can be read publically
4 kkonline 10:08 am Aug 24, 2007
robots.txt="noindex,follow"?
2 davidcamus 9:55 am Aug 24, 2007
how to block a page
on a particular location
7 experienced 9:47 am Aug 24, 2007
Multiple "Sitemap: " lines in robots.txt?
Several TLDs use same site code / robots.txt.
2 geoffyp 1:33 pm Aug 10, 2007
Baiduspider and multiple user-agent text file.
4 Vimes 12:04 pm Aug 7, 2007
How can you Disallow a Sub-Domain?
2 zammo 10:11 am Jul 31, 2007
Google Exclude .swf Files
How do i exclude swf files from indexing by google
5 nxfx 5:26 pm Jul 30, 2007
How to disallow entire content, except a particular location
2 brakkar 12:07 pm Jul 30, 2007
disallowing url midway
2 defanjos 5:20 pm Jul 29, 2007
Block sub directory
5 maxbear 5:24 am Jul 28, 2007
Does this work?
disallow multiple bots at once...
3 Devilfear 6:48 am Jul 27, 2007
Using multiple hash marks
Does that conform to protocol?
5 Reno 7:17 pm Jul 25, 2007
Disallowing Iframed pages
2 Ozz13 1:29 pm Jul 23, 2007
Proper format for disallow of URL
4 Ozz13 1:14 pm Jul 23, 2007
What the dilly?
3 tonynoriega 5:39 pm Jul 18, 2007

Featured Threads

Hot Threads This Week

Hot Threads This Month