Welcome to WebmasterWorld Guest from 107.22.14.254

Forum Moderated by: goodroi

Sitemaps, Meta Data, and robots.txt


Using a robots.txt is all part of being a good SEO. Be sure to check yours in the robots.txt validator that is available to subscribers.

 
Thread SubjectMessagesStarted byLast Message
Robots.txt
5 webboy1 12:04 pm Jul 22, 2002
robots.txt
How to make a valid positive list?
5 matthias 6:20 am Jul 9, 2002
Google - robots.txt - indexing
9 Brett_Tabke 9:40 am Jun 24, 2002
Do I need a robots.txt file for Googlebot to read?
Will I get indexed by Google if I don't have a robots.txt?
27 johnd 10:03 pm Jun 18, 2002
Quick tip
Check out your robots.txt[2] ( 1 2 )
31 GoogleGuy 12:34 pm Jun 18, 2002
Do we need robots.txt?
21 irock 12:17 am Jun 11, 2002
robots.txt and subdomains
how do I block SE's from my subdomains
2 onebaldguy 5:51 am Jun 6, 2002
robots.txt question
2 coolshop 2:21 pm May 31, 2002
User agent in robots.txt and browscap.ini...
Why is there a format different?
6 Pushycat 11:39 pm May 30, 2002
robots.txt -- block one user-agent, allow all others
2 luma 1:44 am May 23, 2002
Robot.txt file help
How to create this file
7 beasscr 6:30 pm May 16, 2002
robots.txt
Generates a 404 error in my logs
9 zeus 1:52 pm May 3, 2002
Does robots.txt override meta tag?
8 kira 7:12 pm May 2, 2002
Robots.txt validator updated.
7 Brett_Tabke 6:48 pm Apr 19, 2002
What is robots.txt syntax
22 Alec_Doggone 9:07 am Apr 19, 2002
Submitting robots.txt to google
Would google look sooner for it?
17 Yidaki 8:19 am Apr 19, 2002
Robots.txt Pattern Matching
How Well Does It Work?
5 WebGuerrilla 1:15 pm Mar 15, 2002
Robots.txt Question?
How often does Ink check for the file??
6 MarkHutch 5:11 pm Mar 11, 2002
Do I need a robots.txt file in my directory?
Google Bot - Whats Up with it?
10 jady 9:58 pm Feb 24, 2002
How important is the Robots.txt file now?
....in 2002
29 ScottM 1:16 am Jan 28, 2002
Robots.txt Question
7 mud 3:21 am Jan 17, 2002
I am writing my 1st robots.txt :) - never bothered before
Anyone know any crawlers I should add ??
3 diddlydazz 4:11 am Jan 12, 2002
MSIECrawler not obeying my robots.txt :(
any ideas ?
5 diddlydazz 3:09 am Jan 12, 2002
Robots.txt standard
4 count_zer0 11:21 am Dec 7, 2001
robots.txt
7 Kenny B 1:02 pm Sep 25, 2001

Featured Threads

Hot Threads This Week

Hot Threads This Month