Welcome to WebmasterWorld Guest from 54.145.246.183

Forum Moderated by: goodroi

Sitemaps, Meta Data, and robots.txt


Using a robots.txt is all part of being a good SEO. Be sure to check yours in the robots.txt validator that is available to subscribers.

 
Thread SubjectMessagesStarted byLast Message
Logic Error
Wildcard userAgent last entry in robots.txt file?
2 pageoneresults 2:34 pm May 20, 2004
Question about robots meta tag, as well as robots.txt....
3 xsirrionx 11:48 am May 20, 2004
How to validate robots.txt on "https" site?
Please help me validate my "https" site
2 CodeCrunch 12:29 am May 20, 2004
Wild chars *.* allowed in "Disallow" of Robots.txt?
or anything else like *.txt, etc.
2 CodeCrunch 12:26 am May 20, 2004
robots.txt restrictions conflictual with site's Terms
Which of them should prevail?
4 cristiscu 6:52 am May 19, 2004
Comments at the end of a line bad style, why?
3 zachariah 6:52 am May 18, 2004
Spider Testing
6 cabowabo 3:44 am May 13, 2004
Meta Tags & Robots.txt
4 tcustom 1:57 am May 13, 2004
Why is Google still indexing my SWF files?
2 ichthyous 6:10 am May 10, 2004
Deny or Not to Deny /images
Denying /images folder
9 quebrada 12:19 am May 6, 2004
proper way of changing site
2 jmdb71 5:11 pm May 1, 2004
How to get spider to visit parts of site more frequently
Is this possible?
5 Pimpernel 10:35 am Apr 30, 2004
robots.txt missing?
2 zyshen 5:19 pm Apr 27, 2004
Google ignoring robots.txt
I'm seeing lots of my 'Disallowed' pages in their index
4 Amygdala 4:54 am Apr 24, 2004
Strange bot keep on visiting my site and don't obey my robots.txt file
What to do?
2 Compworld 8:01 pm Apr 22, 2004
revisit after so many days
2 tommygunner 3:21 am Apr 21, 2004
index.php vs index.html
Will This Work i Wonder
2 yummycarp 11:28 pm Apr 20, 2004
Banning a script using robots.txt
how do do this
4 Imaster 9:51 pm Apr 20, 2004
Robots.txt - Why?
3 warlordbb 2:08 pm Apr 20, 2004
Can IMDB do this?
charge per unwanted request
3 jrap 10:19 pm Apr 19, 2004
Robots.txt exclusion
Google keeps crawling page
2 anxvariety 2:27 pm Apr 19, 2004
Froogle wants my images!
3 seadog 5:59 am Apr 14, 2004
META Robots no index tag
2 BMach 2:36 am Apr 14, 2004
Using wild card in robots
Can you add * to ip?
2 walrus 5:37 pm Apr 13, 2004
How to ban inktomisearch.com's bots?
I have been hit hard with so many requests by this.
3 itisgene 5:24 pm Apr 11, 2004

Featured Threads

My Threads

Hot Threads This Week

Hot Threads This Month