Welcome to WebmasterWorld Guest from 18.104.22.168
Using a robots.txt is all part of being a good SEO. Be sure to check yours in the robots.txt validator that is available to subscribers.
|Thread Subject||Messages||Started by||Last Message|
|Blocking image file indexing without robots.txt
Can this be done?
|2||Winedark||8:08 pm Oct 28, 2003|
How long can a robots.txt be?
|4||WyrdoManx||9:47 pm Oct 22, 2003|
|Robots.txt - Am I Missing Somthing?
Query strings and general syntax is suspected...
|13||Nick_W||3:40 pm Oct 21, 2003|
|How to correctly set robots.txt for a subdomain?
|5||PatrickDeese||6:36 pm Oct 15, 2003|
|Robots.txt for Query Strings
Can I stop bots from following links with query stings
|10||Nick_W||6:08 pm Oct 15, 2003|
Crawling no longer allowed
|12||Yidaki||5:38 pm Oct 14, 2003|
|Robots.txt Question :)
Question regarding how to use robots txt
|8||Prana||9:33 pm Sep 27, 2003|
3 domains in root
|9||Alternative_Future||11:33 pm Sep 23, 2003|
|Robots.txt disallow: /index.php?
then /index.php?param=example still allowed?
|5||Yidaki||12:17 pm Sep 21, 2003|
|Robots.txt exclusion question
Robots.txt exclude all but one directory?
|4||olof_bailey||8:52 pm Sep 20, 2003|
Where does it go?
|4||zoobie||5:58 pm Sep 19, 2003|
|404 robots.txt and traffic going down
||8||silverbytes||4:29 pm Sep 14, 2003|
||4||johnnstacy1||8:19 am Sep 1, 2003|
|More than one Robots.txt per server?
||18||apphoto||6:05 pm Aug 30, 2003|
|robots.txt case sensitive?
||3||the_nerd||7:48 am Aug 19, 2003|
|robots.txt expert question
Is it possible for robots.txt to disallow all but one page?
|4||DanielDiGriz||11:38 pm Aug 16, 2003|
|querystrings and robots.txt
||1||driesie||10:06 am Aug 13, 2003|
|Fav icon and robots.txt
Do I need these?
|7||spikedo55||3:49 pm Aug 12, 2003|
|robots.txt - allow folder withing disallowed one
||3||moltar||5:45 am Aug 10, 2003|
|robots.txt Disallow - How much time till its obeyed?
||6||notsleepy||11:17 pm Aug 6, 2003|
|robots.txt syntax question
Block access to specific file
|2||bcolflesh||8:19 pm Aug 2, 2003|
|Robots.txt on phpbb forums
it worked! thanks for the tip
|1||dvduval||1:52 pm Jul 29, 2003|
Is this valid
|6||customdy||5:01 pm Jul 16, 2003|
|Is there a "best" robot txt to get all pages ranked?
What is very best robot txt coding to insure Google spider index all pages?
|5||EandA||1:13 pm Jul 13, 2003|
|Really necessary to add to robots.txt
||2||coolshop||9:54 pm Jul 4, 2003|