Welcome to WebmasterWorld Guest from 54.144.79.200

Forum Moderated by: goodroi

Sitemaps, Meta Data, and robots.txt


Using a robots.txt is all part of being a good SEO. Be sure to check yours in the robots.txt validator that is available to subscribers.

 
Thread SubjectMessagesStarted byLast Message
Should this file be "Public"
Is it good practice to have the robots.txt file.......
4 Propools 11:58 am Sep 26, 2007
Proper URL & File Structure
Fighting with Robots. Txt
3 srkmodi 12:34 pm Sep 24, 2007
Use & or & amp; in robots.txt
Which to use?
3 g1smd 1:53 pm Sep 21, 2007
Using allow to index search-only pages
3 timmbbo 12:31 pm Sep 18, 2007
Disallow - wait a day - add pages
1 brevetoxin 1:36 am Sep 12, 2007
Robots.txt to improve ranks
Will blocking of internal pages through robots.txt improve SE rankings
3 pro_seo 5:18 am Sep 10, 2007
Robots INclusions standard?
13 mikomido 12:28 am Sep 9, 2007
blocking the hostsite but not the hosted sites
3 messemaeker 12:12 pm Sep 7, 2007
Blocking Windows Live robot/spider.
2 Tidal2 10:56 am Sep 7, 2007
Do we have to use robots.txt on every website?
for robots.txt issues
5 masood82 8:42 am Sep 6, 2007
How to block tags?
hundreds of undesired links !
3 rashe18 4:33 pm Sep 4, 2007
Blocked through robots.txt . will the page be crawled ever
5 getxb 6:02 am Aug 31, 2007
URI Only Listing
Need a refresher course on this...
9 pageoneresults 12:20 pm Aug 24, 2007
robot.txt Security
Can be read publically
4 kkonline 10:08 am Aug 24, 2007
robots.txt="noindex,follow"?
2 davidcamus 9:55 am Aug 24, 2007
how to block a page
on a particular location
7 experienced 9:47 am Aug 24, 2007
Multiple "Sitemap: " lines in robots.txt?
Several TLDs use same site code / robots.txt.
2 geoffyp 1:33 pm Aug 10, 2007
Baiduspider and multiple user-agent text file.
4 Vimes 12:04 pm Aug 7, 2007
How can you Disallow a Sub-Domain?
2 zammo 10:11 am Jul 31, 2007
Google Exclude .swf Files
How do i exclude swf files from indexing by google
5 nxfx 5:26 pm Jul 30, 2007
How to disallow entire content, except a particular location
2 brakkar 12:07 pm Jul 30, 2007
disallowing url midway
2 defanjos 5:20 pm Jul 29, 2007
Block sub directory
5 maxbear 5:24 am Jul 28, 2007
Does this work?
disallow multiple bots at once...
3 Devilfear 6:48 am Jul 27, 2007
Using multiple hash marks
Does that conform to protocol?
5 Reno 7:17 pm Jul 25, 2007

Featured Threads

Hot Threads This Week

Hot Threads This Month