Forum Moderated by: goodroi
Using a robots.txt is all part of being a good SEO.
| Thread Subject | Messages | Started by | Last Message | ||||
|---|---|---|---|---|---|---|---|
| Robot.txt file deleted by accident what now? |
2 | Hoppo | 11:15 pm Dec 12, 2004 | ||||
| robots.txt and meta tags can I use both? |
2 | kdawg | 11:23 pm Dec 11, 2004 | ||||
| Oddly long robots.txt file Seeking explanation for weird robots.txt file |
7 | bendede | 9:31 pm Dec 9, 2004 | ||||
| Hidden Text? Assistance Please.... |
4 | Kasey22 | 8:44 pm Dec 7, 2004 | ||||
| Google wildcard Is this allowed? |
3 | maha | 5:02 am Dec 6, 2004 | ||||
| How do I create two robots.txt for the same directory? .com and .net are for two different engines but point at the same folder |
3 | walkman | 9:01 pm Dec 2, 2004 | ||||
| Robots.txt Newbie Question |
4 | paulpsd | 12:01 am Dec 2, 2004 | ||||
| redirecting domain |
3 | tomasz | 9:31 pm Dec 1, 2004 | ||||
| subdomains and robots.txt |
2 | gettin_there | 8:54 pm Dec 1, 2004 | ||||
| User-agent: ... |
3 | gettin_there | 3:06 pm Dec 1, 2004 | ||||
| Crawling of non-linked pages Over 18's area |
3 | theleveller | 2:43 pm Dec 1, 2004 | ||||
| Ask Jeeves Question Need to ban a bot |
2 | Marcus_Aurelius | 10:13 am Dec 1, 2004 | ||||
| Webmaster World's Robots.txt Why don't they allow msnbot? |
6 | Rollo | 6:11 pm Nov 30, 2004 | ||||
| Content of Robots.txt files You wouldn't believe what some hosts think should be in a Robots.txt file! |
9 | AlucardSpyderWriter | 8:17 pm Nov 28, 2004 | ||||
| How would I disallow Google... ...and allow all others? |
3 | Rollo | 5:35 pm Nov 28, 2004 | ||||
| Robot.txt specialists: will Disallow: /? work? need to block google from ...com/?tracking. |
1 | walkman | 7:46 am Nov 28, 2004 | ||||
| robots.txt-newbie needs help Is this correct? |
4 | cybervette | 5:18 pm Nov 26, 2004 | ||||
| Changing your Mind with robots.txt Will googlebot come back to the party? |
2 | Spine | 4:41 pm Nov 26, 2004 | ||||
| dynamic folders |
1 | apan | 12:24 pm Nov 25, 2004 | ||||
| robots.txt validator problem? it is reading my index.htm file? |
2 | mbatta | 11:34 pm Nov 22, 2004 | ||||
| Syntax: User-agent: Mediapartners-Google* |
5 | jhubb | 3:21 pm Nov 22, 2004 | ||||
| robots.txt question I asked whether does this, but the engine spider ip address has any to use |
2 | ericchui | 11:18 am Nov 22, 2004 | ||||
| Protecting robots.txt file from bad guys Is it enough to passwoed protect those dirs? |
3 | damlag | 4:10 pm Nov 21, 2004 | ||||
| How to Exclude from Yahoo/MSN results! I want to exclude underconstruction site |
2 | fm101 | 1:49 am Nov 21, 2004 | ||||
| How to block all subdirectories? |
2 | TerryMc | 7:05 pm Nov 17, 2004 |
