Welcome to WebmasterWorld Guest from 54.162.167.40

Forum Moderated by: goodroi

Sitemaps, Meta Data, and robots.txt


Using a robots.txt is all part of being a good SEO. Be sure to check yours in the robots.txt validator that is available to subscribers.

 
Thread SubjectMessagesStarted byLast Message
Fav icon and robots.txt
Do I need these?
7 spikedo55 3:49 pm Aug 12, 2003
robots.txt - allow folder withing disallowed one
3 moltar 5:45 am Aug 10, 2003
robots.txt Disallow - How much time till its obeyed?
6 notsleepy 11:17 pm Aug 6, 2003
robots.txt syntax question
Block access to specific file
2 bcolflesh 8:19 pm Aug 2, 2003
Robots.txt on phpbb forums
it worked! thanks for the tip
1 dvduval 1:52 pm Jul 29, 2003
robots.txt
Is this valid
6 customdy 5:01 pm Jul 16, 2003
Is there a "best" robot txt to get all pages ranked?
What is very best robot txt coding to insure Google spider index all pages?
5 EandA 1:13 pm Jul 13, 2003
Really necessary to add to robots.txt
2 coolshop 9:54 pm Jul 4, 2003
robots.txt file
3 ktd121 3:14 am Jul 4, 2003
meta name = robot
What does that do?
7 aroye 5:57 pm Jun 30, 2003
Scooter 3.3 ignores robots.txt?
this is ridiculous
6 berli 2:39 am Jun 27, 2003
Robots.txt
Help please mine is missing?
3 peteredmonds 8:37 pm Jun 25, 2003
Robots.txt
Huh?
2 zoobie 6:52 pm Jun 19, 2003
"robots"
What should it be set at?
4 kimderella86 6:22 pm Jun 18, 2003
Robots.txt exclusion for dynamic page
2 rogerd 9:22 pm Jun 17, 2003
Robots.txt & How your site is spidered
Positives of Robots.txt file aside from disallowing bots
4 floridadesigns 11:31 pm Jun 14, 2003
Why should I obey robots.txt
Developer questions
22 PWalker 1:01 am Jun 13, 2003
Php robots.txt parser script
does that exitst? if not regex help needed.
5 ikbenhet1 2:57 pm Jun 9, 2003
files requested by robots.txt
i would like to find out about the files requested by robots
8 andrea_edwards 7:53 pm Jun 8, 2003
Toolbar and Robots.txt
8 rintrah 8:41 pm Jun 5, 2003
Robots.txt - files in folders
3 rintrah 9:43 pm Jun 4, 2003
robots.txt
bad spider
6 copongcopong 9:11 pm Jun 4, 2003
Robots.txt - What's the Point?
7 rintrah 6:59 pm May 30, 2003
Pragma & Cache-Control
Are these Safe with Search Engines
9 stuart 8:32 pm May 27, 2003
Quick robots.txt question
Do I just need to put the file name
2 bateman_ap 8:21 pm May 24, 2003

Featured Threads

Hot Threads This Week

Hot Threads This Month