Welcome to WebmasterWorld Guest from 23.20.137.66

Forum Moderated by: goodroi

Sitemaps, Meta Data, and robots.txt


Using a robots.txt is all part of being a good SEO. Be sure to check yours in the robots.txt validator that is available to subscribers.

 
Thread SubjectMessagesStarted byLast Message
robots.txt-newbie needs help
Is this correct?
4 cybervette 5:18 pm Nov 26, 2004
Changing your Mind with robots.txt
Will googlebot come back to the party?
2 Spine 4:41 pm Nov 26, 2004
dynamic folders
1 apan 12:24 pm Nov 25, 2004
robots.txt validator problem?
it is reading my index.htm file?
2 mbatta 11:34 pm Nov 22, 2004
Syntax: User-agent: Mediapartners-Google*
5 jhubb 3:21 pm Nov 22, 2004
robots.txt question
I asked whether does this, but the engine spider ip address has any to use
2 ericchui 11:18 am Nov 22, 2004
Protecting robots.txt file from bad guys
Is it enough to passwoed protect those dirs?
3 damlag 4:10 pm Nov 21, 2004
How to Exclude from Yahoo/MSN results!
I want to exclude underconstruction site
2 fm101 1:49 am Nov 21, 2004
How to block all subdirectories?
2 TerryMc 7:05 pm Nov 17, 2004
How to exclude dynamic URLS?
So Search engines dont get bogged down.
7 Tech2004 4:54 am Nov 16, 2004
Effect of Robots.txt
5 seolancer 5:31 am Nov 15, 2004
robots.txt - newbie needs help
5 geezer_uk 5:39 am Nov 10, 2004
excluding cgi-bin
is this right
2 rbacal 2:59 am Nov 10, 2004
Is robot.txt a must for good SEO?
--whats the purpose of robot.txt
4 redleaf 12:24 am Nov 9, 2004
Weird 404 error hitting on robot.txt
anyone experienced this before?
3 redleaf 9:30 am Nov 8, 2004
robots.txt and free web hosting
3 rhodopsin 7:46 am Nov 8, 2004
Possible to have a page disalowed by robots.txt but indexed by adsense
Possible to do this?
2 brakkar 9:16 pm Nov 5, 2004
Protocol interpretation
Peerbot or I misunderstood the robot exclusion protocol
3 WebJoe 9:11 pm Nov 5, 2004
Need help about File Robots.txt
how do I know spiders visit to my site or not?
3 viva 10:15 am Nov 4, 2004
what happen with robot exclusion and iframes?
could it be a problem to exclude content that appears inside a home iframe?
2 mgorricho 5:03 pm Nov 3, 2004
protocol extension resource control
proposed extension to robots.txt protocol allowing resource control
1 copsewood 11:29 am Nov 2, 2004
Querystring check
2 papamaku 9:36 am Nov 2, 2004
robots and nav bars
excluding links in nav bars ...
2 f00fighter 11:38 pm Oct 29, 2004
Need assistance with robot controls
need help with a robot .txt file
2 borkboing 11:31 pm Oct 29, 2004
Robots.txt not working
bots not obeying robots.txt
4 cyberjunkie 11:05 pm Oct 29, 2004

Featured Threads

Hot Threads This Week

Hot Threads This Month