Welcome to WebmasterWorld Guest from 23.20.241.155

Forum Moderated by: goodroi

Sitemaps, Meta Data, and robots.txt


Using a robots.txt is all part of being a good SEO. Be sure to check yours in the robots.txt validator that is available to subscribers.

 
Thread SubjectMessagesStarted byLast Message
Does excluding Googlebot create a penalty?
If other bots are allowed, is this a red flag to Google
2 AndyA 9:26 pm Nov 27, 2006
what to put into robots.txt?
11 Hitman3266 5:09 pm Nov 27, 2006
How long does it take for SEs to learn about a robots'txt
4 bouncybunny 12:04 pm Nov 20, 2006
Yahoo! Slurp Now Supports Wildcards in robots.txt
17 OldWolf 2:54 am Nov 20, 2006
Robots.txt and password-protection
3 hwy2hel 10:56 pm Nov 16, 2006
Yahoo robots txt
nedd to disallow all urls with?
2 cheaperholidays 4:54 pm Nov 16, 2006
robots.txt? OR .htaccess?
disallowing or protecting directories
2 johnjoyce 4:27 pm Nov 16, 2006
Wildcards in robots.txt for forum software
6 Asia_Expat 1:40 am Nov 14, 2006
robots.txt syntax
'$' and '?'
2 sssweb 8:40 pm Nov 11, 2006
robots.txt help
2 miracle 7:16 pm Nov 8, 2006
Link Checker / Site map generator
One that observes robots.txt and meta tags
5 fish_eye 5:33 am Nov 7, 2006
Disallow by ISP
Where to get a list of ISPs and the countries they are in?
1 youfoundjake 7:13 pm Nov 3, 2006
Made the mistake of putting sub folders I want allowed
disallow everthing but?
2 Sunnyvale 7:07 pm Oct 31, 2006
Google remove url tool for supplemental urls a waste of time?
Should I bother removing duplicate supplementals?
3 Tonerman 4:20 pm Oct 30, 2006
allow major engines, block the fleas
and to add further control
5 youfoundjake 5:16 pm Oct 26, 2006
Dissallowing Dynamic Content
Blocking "index.php?"
2 trillianjedi 6:58 pm Oct 25, 2006
Combining htaccess and robots.txt?
7 Lady18 11:37 pm Oct 24, 2006
disallowing a 'staging server'
2 rden17 4:11 pm Oct 24, 2006
.htaccess
2 ssgumby 9:53 pm Oct 21, 2006
cloaking robots.txt
to avoid hacking
4 idolw 12:26 pm Oct 18, 2006
How to deal with a robots.txt hack attack
Guidelines to mitigate the damage from an attack
3 Whitey 10:17 pm Oct 17, 2006
Mainseek bot
this bot is ripping through our site
2 ssgumby 6:36 pm Oct 14, 2006
Google indexing weird links
ones with // in them
4 netchicken1 12:54 am Oct 13, 2006
Please welcome your new moderator: goodroi
20 engine 6:35 am Oct 11, 2006
robots.txt problem and subdirs / wildcards
2 disgust 1:44 pm Oct 10, 2006

Featured Threads

Hot Threads This Week

Hot Threads This Month