Welcome to WebmasterWorld Guest from 23.22.46.195

Forum Moderated by: goodroi

Sitemaps, Meta Data, and robots.txt


Using a robots.txt is all part of being a good SEO. Be sure to check yours in the robots.txt validator that is available to subscribers.

 
Thread SubjectMessagesStarted byLast Message
Disallow /file.php same as Diallow file.php?
6 One_on_One 3:43 am Jun 2, 2005
Stopping spiders
from Competitors Analysis Tools...
4 ambiorix_rb 1:21 pm Jun 1, 2005
SEO / robots.txt question
unintentional cross-linking penalty
1 wolfgang 6:27 pm May 31, 2005
Blocking a Sub Domain Directory
3 is300 8:25 am May 30, 2005
Order allow,deny or deny,allow
Not sure which order is best...
3 whoisgregg 9:16 pm May 27, 2005
Search Engine Crawlers
2 dayo 8:39 pm May 26, 2005
Spidering of my code pages
Wrong pages being spidered and losing referals
5 djfelip 11:18 pm May 25, 2005
robots4.txt -- I'm confused
Confused about what is allowed in this file
3 Goose_68 8:46 pm May 24, 2005
newbie question
http://www.yourdomain.com/robots.txt
2 webmz 3:10 am May 22, 2005
Validation
18 Dexie 4:14 pm May 21, 2005
Can't find the robot.txt
where is it?
27 mmmwowmmm 10:56 am May 21, 2005
SonicWALL Firewall Blocks Spiders
Settings will block spiders from indexing site.
26 EliteWeb 3:16 pm May 15, 2005
Robots.txt Format
Will this work?
4 kbba04527 6:37 am May 6, 2005
googlebot, robots.txt and spidering
Perplexed by lack or spidering of the site
8 partnermine 5:12 am May 6, 2005
meta tags+robots.txt
newbie question
3 bonkph 8:16 pm May 5, 2005
Baiduspider - not checking robots.txt & how to block?
6 TreeShare 8:07 am May 5, 2005
robots.txt and subdomain
Can I use this file to block a specific subdomain
4 Pat1975 8:57 am May 4, 2005
google hits robots.txt but does not index anything?
am i doing something wrong?
19 mbatta 5:38 am May 4, 2005
Can someone tell me how to ........
3 Lex_Luther 9:00 pm May 3, 2005
Google - User Agent: * Disallow: /
4 wchan07 4:42 pm May 3, 2005
mod_rewrite and robots.txt
2 jewels 3:03 pm May 3, 2005
Blocking Yahoo from a specific folder
How do I do this?
2 Rollo 2:55 pm May 3, 2005
How to Validate robots.txt and do I even have one?
13 neophitemoneymaker 11:58 pm May 1, 2005
Page indexed in Google despite robots.txt entry
How did they find it in the first place?
23 pmkpmk 9:17 pm May 1, 2005
The Great SearchEngineWorld robots.txt Survey #2 is Under Way
9 Brett_Tabke 2:35 pm Apr 29, 2005

Featured Threads

My Threads

Hot Threads This Week

Hot Threads This Month