Welcome to WebmasterWorld Guest from 23.22.46.195

Forum Moderated by: goodroi

Sitemaps, Meta Data, and robots.txt


Using a robots.txt is all part of being a good SEO. Be sure to check yours in the robots.txt validator that is available to subscribers.

 
Thread SubjectMessagesStarted byLast Message
Dmoz robots.txt
Is the dmoz.org/world disallowed?
4 angiolo 8:05 am Aug 6, 2005
How to exclude Oracle Portal Help pages using robots.txt
Need to exclude Oracle Portal Help Pages from search engines
10 atul_iiit 9:12 am Aug 5, 2005
http://<hostname.domainname>/robots.txt is access denied can robots
Can robots read robots.txt if I get access denied for this file
4 atul_iiit 4:25 pm Aug 4, 2005
What's Robot.txt?
What's the importance of Robot.txt?
3 Lowkei 9:02 am Jul 31, 2005
Don't want to index dynamic pages
But would it be a mistake to block them?
5 chopin2256 2:12 am Jul 30, 2005
Could 744 permissions on robots.txt cause problems
all other files are set to 644
5 dbar 7:41 pm Jul 29, 2005
Robots.txt viewable
viewing the robots.txt file
2 eleap 3:27 pm Jul 27, 2005
How to exclude a few pages?
2 jack38 10:25 am Jul 27, 2005
Blocking a file extension with robots.txt
Inherited a site with a problem
5 abbeyvet 5:45 pm Jul 26, 2005
How to avoid getting print-pages indexed?
What's the best way for Google?
1 simczak 3:42 pm Jul 24, 2005
Robots.txt stops automated link software
2 jdhuk 11:01 pm Jul 18, 2005
Which editor?
Notepad or editor in UNIX
3 Bali_Freak 12:15 pm Jul 18, 2005
selectively including folders
that are beneath a folder excluded
2 stu2 5:20 am Jul 17, 2005
Would this work? Robots exclusion or considered cloaking?
5 dvldvl3 5:11 am Jul 17, 2005
Previously Indexed Files-Want To Exclude Now
RobotsTxt Change
3 buview1 12:23 pm Jul 16, 2005
Google only hits robots and index
Google only hits robots and index
10 rostom 3:51 pm Jul 15, 2005
Disallowing & in url
2 Wolvereness 2:54 pm Jul 12, 2005
robots.txt verses sitemap.html verses OPML file
2 Neal 8:39 pm Jul 11, 2005
Just added robots.txt file
4 soccrt 4:41 am Jul 11, 2005
block all except
2 Radial 9:31 pm Jul 7, 2005
robots.txt with? in URL
16 jozomannen 11:17 am Jul 6, 2005
How do i add robot.txt?
10 genny 4:38 am Jul 6, 2005
Include files, spiders, and robots.txt
4 rottsnroses 11:24 pm Jul 3, 2005
How to Stop BecomeBot?
12 shilmy 3:09 pm Jun 29, 2005
Google ignoring my robots.txt?
9 homesby 12:17 am Jun 29, 2005

Featured Threads

My Threads

Hot Threads This Week

Hot Threads This Month