robots.txt |
| Forum Charter, Library, Moderated by: ThomasB & goodroi |
Forum Options:
Reset Last Read Pointers |
Using a robots.txt is all part of being a good SEO. Be sure to check yours in the robots.txt validator that is available to subscribers.
|
| |
|
| new |
Subject |
Messages | Started by |
Last Message |
 |
4 Wordpress File's Link To Be Blocked
How should I block them?
| 3
|
aced84 | 11:13 am on May 17, 2008 (utc 0)
|
|
|
I think there is a problem
Robots problem
| 2
|
AllanS | 10:39 am on May 17, 2008 (utc 0)
|
|
|
Blocking parts of forum
but not others
| 10
|
AnonyMouse | 7:13 pm on May 16, 2008 (utc 0)
|
|
|
robots.txt syntax for wildcards?
| 3
|
Marfola | 7:15 am on May 15, 2008 (utc 0)
|
|
|
How many links can i place in one page of site map?
| 5
|
bava_seo | 2:12 pm on May 10, 2008 (utc 0)
|
|
|
why use and what is robots.txt
this maybe stupid question
| 3
|
mick_simon | 4:45 pm on May 7, 2008 (utc 0)
|
|
|
How To stop crawler to crawl https:// pages
| 8
|
vreelmistee1 | 6:03 am on May 7, 2008 (utc 0)
|
|
|
Status: 404(not found) in Google Webmaster Tools
| 3
|
Boulder90 | 6:39 pm on May 6, 2008 (utc 0)
|
|
|
Y! and robots.txt
Partial ignore
| 5
|
Staffa | 6:36 pm on May 6, 2008 (utc 0)
|
|
|
Googlebot is Not Taking Visit to my site
robots.txt is restricted googlebot to do not visit
| 4
|
bilalseo | 1:54 pm on May 6, 2008 (utc 0)
|
|
|
Site Pages are Blocked by Robots.txt
| 4
|
bilalseo | 10:03 am on May 6, 2008 (utc 0)
|
|
|
robots.txt and IP addresses
| 11
|
Maciej_Ziemczonek | 10:11 pm on May 2, 2008 (utc 0)
|
|
|
Is this valid? wildcard in robots
Disallow: /*/edit.html
| 3
|
rudyten | 1:47 pm on April 20, 2008 (utc 0)
|
|
|
Will spiders follow links on URL's blocked by robots.txt
| 3
|
deanw | 6:34 pm on April 16, 2008 (utc 0)
|
|
|
Dynamic URL's and robots.txt
| 3
|
cgydosh | 8:16 pm on April 13, 2008 (utc 0)
|
|
|
spiders list
| 2
|
hugh | 12:56 pm on April 9, 2008 (utc 0)
|
|
|
Security Challenges
| 4
|
pageoneresults | 3:49 pm on April 2, 2008 (utc 0)
|
|
|
Dumbest question eva: Do all major SEs support the Google guidelines?
I am testing mine on G's WebmasterCentral
| 2
|
walkman | 6:42 pm on Mar. 26, 2008 (utc 0)
|
|
|
Is there any way to block everyone except.
for Google, Yahoo etc?
| 2
|
mmiller | 9:41 am on Mar. 21, 2008 (utc 0)
|
|
|
need to block. and don't know - pls help
need help with blocking - Malware Ref
| 4
|
Reilly | 7:55 pm on Mar. 17, 2008 (utc 0)
|
|
|
How to block original + localized pages?
robots.txt localized
| 1
|
vanguard2000 | 6:19 pm on Mar. 17, 2008 (utc 0)
|
|
|
How long before bots re-visit after they were told they are not allowed?
| 3
|
boromir27 | 3:32 am on Mar. 17, 2008 (utc 0)
|
|
|
Inktomi User-Agent blocked
Inktomi blocked
| 6
|
cyberdyne | 8:36 am on Mar. 14, 2008 (utc 0)
|
|
|
robots.txt - Disallow/Allow
| 7
|
eyalkattan | 1:14 am on Mar. 13, 2008 (utc 0)
|
|
|
robots and redirecting
i am wondering how a robot would act on a page with a redirect
| 2
|
melanger | 8:56 pm on Mar. 8, 2008 (utc 0)
|
|
|
Unusual log entries
| 5
|
cyberdyne | 6:17 pm on Mar. 8, 2008 (utc 0)
|
|
|
Redirect robot.txt from http to https
how to redirect properly.
| 8
|
clyde4210 | 2:59 pm on Mar. 6, 2008 (utc 0)
|
|
|
Uh oh, it's happening again
robots.txt blocked out
| 2
|
wesg | 9:58 pm on Feb. 28, 2008 (utc 0)
|
|
|
new badbot: EnaBot/1.0
| 7
|
Achernar | 8:31 am on Feb. 28, 2008 (utc 0)
|
|
|
Need help blocking search results page
Lots of keyword search pages indexed, some with no content
| 4
|
ecmedia | 2:28 pm on Feb. 26, 2008 (utc 0)
|
|
|
disallow ALL bots.and then allow the good ones?
| 5
|
tokey666 | 8:40 pm on Feb. 25, 2008 (utc 0)
|
|
|
Excluding a stub page with robots.txt
Dealing with trailing forward slash
| 4
|
bouncybunny | 12:41 am on Feb. 24, 2008 (utc 0)
|
|
|
What is wrong with my Robots.txt file?
I'd appreciate help with my Robots.txt file
| 7
|
Morthan | 3:07 am on Feb. 22, 2008 (utc 0)
|
|
|
Help with blocking URLS
| 5
|
jaffstar | 7:17 am on Feb. 18, 2008 (utc 0)
|
|
|
301 Redirect and Search Ranking
301 Redirect and Search Ranking
| 4
|
bilalseo | 6:22 am on Feb. 18, 2008 (utc 0)
|
|
|
No life without robots.txt file
This may be old hat to you but it knocked me for six
| 18
|
folkranger | 7:17 pm on Feb. 15, 2008 (utc 0)
|
|
|
Robots.txt and iframes
How to disallow iframes using robots.txt?
| 6
|
foxtunes | 11:00 pm on Feb. 14, 2008 (utc 0)
|
|
|
How to Handle Cname URL
Indexed in Google
| 7
|
experienced | 7:47 am on Feb. 12, 2008 (utc 0)
|
|
|
Google problem with robots.txt
Webmaster tools isn't helping.
| 21
|
wesg | 8:16 pm on Feb. 5, 2008 (utc 0)
|
|
|
*/? */index.*? is a must in robots.txt
I see /?widgetid=45 indexed
| 3
|
AjiNIMC | 8:17 am on Jan. 30, 2008 (utc 0)
|
|
|
Robots.txt (/index.jsp?pg name=ccpm/lob/sublob/pgname&site=a1)
Pattern on parameters
| 2
|
Surendra | 12:20 am on Jan. 29, 2008 (utc 0)
|
|
|
Denying access to everything but the home page?
Using robots.txt to only allow robot access to the home page
| 3
|
monkeylytics | 8:00 pm on Jan. 28, 2008 (utc 0)
|
|
|
Control Robots with Meta or robots.txt
| 3
|
driller41 | 12:21 pm on Jan. 15, 2008 (utc 0)
|
|
|
Robots.txt disallowed.
| 12
|
g1smd | 12:18 am on Jan. 15, 2008 (utc 0)
|
|
|
robots.txt with wildcard?
| 3
|
tranceformer | 12:39 am on Jan. 13, 2008 (utc 0)
|
|
|
Disallowing with robots.txt
robots.txt, disallow certain urls
| 3
|
jim_knopf | 12:36 am on Jan. 13, 2008 (utc 0)
|
|
|
Getting correct user agent
getting from access log the correct user agent for robots.txt
| 2
|
hessodreamy | 9:03 pm on Jan. 11, 2008 (utc 0)
|
|
|
Scrapers checking robots.txt
Spiders arrived on heels of robots.txt deployment
| 4
|
timster | 1:29 am on Jan. 4, 2008 (utc 0)
|
|
|
404 Errors in Webmaster Console
What to do with these pages?
| 2
|
rowtc2 | 11:35 pm on Jan. 3, 2008 (utc 0)
|
|
|
directory and subdirectory access
allowing access to subdirectory but not main directory
| 3
|
Mr_Cool_Dude | 6:11 pm on Jan. 2, 2008 (utc 0)
|
| Displaying Page 1 of 24 |
[ 1 ] 2 3 4 5 6 7 8 9 10 ... 24 Next >> |
=New messages indicator
|
