Welcome to WebmasterWorld Guest from 54.160.163.163

Forum Moderated by: goodroi

Sitemaps, Meta Data, and robots.txt


Using a robots.txt is all part of being a good SEO. Be sure to check yours in the robots.txt validator that is available to subscribers.

 
Thread SubjectMessagesStarted byLast Message
Insert disallow.php to robots
is it a good idea?
3 sirkei 5:52 am Oct 3, 2004
Can i add robot txt to a free hosting site? keywords
2 alze 5:14 am Oct 3, 2004
Does robots.txt prevent all access?
...on page as well as directory.
2 yump 5:02 am Oct 3, 2004
Robots.txt sabotage
Common? How to protect?
4 adfree 11:05 am Sep 30, 2004
newbie to robots.txt, want to exclude /images/
excluding some image folders but not others
2 franco81 3:29 am Sep 30, 2004
Newbie - Why wouldn't I want to allow certain robots?
If I don't have anything on the site I want to keep private
4 AmyG 4:03 pm Sep 29, 2004
What if your domains robots.txt
Was pointed to your domain?
5 Daily_Sparring 3:22 am Sep 27, 2004
Where to put robots in website?
I has doubt where to put robots.
3 viva 10:00 am Sep 26, 2004
robots.txt and .htaccess
1 HuhuFruFru 12:02 am Sep 26, 2004
Excluding Sub Folders
When you exclude a folder, are folders inside that excluded also?
3 damlag 8:39 pm Sep 25, 2004
Can someone help me write a robots.txt?
5 budbiss 11:50 pm Sep 19, 2004
www.example.com/view.php
How to avoid spidering of anything with this prefix
2 nuevojefe 10:26 pm Sep 19, 2004
robots.txt
Invalid warnings
5 wolfline 5:10 am Sep 17, 2004
possible sabotage?
robot.txt working against me?
5 seofan 9:36 pm Sep 8, 2004
who can give me a website log file sample to see?
website log file
3 Tellist 5:19 pm Sep 4, 2004
Google CHTML Proxy disobeying robots.txt
Why?
5 yosmc 12:16 pm Aug 31, 2004
Google tells me to change my robots.txt
Should I follow the advice from Google adSense?
2 Bonusbana 2:44 pm Aug 27, 2004
Master List of Malicious Robots
Is there an active list of the bots we should block?
5 Eric_Lander 4:09 pm Aug 25, 2004
robots.txt security problems?
2 helohelo 3:44 pm Aug 25, 2004
301 Redirect domain?
How to handle robots.txt?
5 Mr_Roberto 1:13 am Aug 24, 2004
Creating robots.txt file.
6 kabanero 8:34 pm Aug 18, 2004
Blocking just one parameter via robots.txt
Is this even possible?
2 werty 7:31 pm Aug 18, 2004
Problem with spiderbots scrounging for robot.txt
3 Mr_N 6:49 am Aug 10, 2004
i have a subdomain I dont want indexed
is robots.txt the answer?
4 chrisandsarah 1:21 pm Aug 8, 2004
robots.txt appearing in my log files
But I have n't got one there!
5 uk_webber 2:59 pm Aug 5, 2004

Featured Threads

Hot Threads This Week

Hot Threads This Month