WP blog traffic plummeting. Bad robots.txt?

[Boulder90]

Thanks for any help on this one.

My hobby blog gets about 1k visitors a day and on April 18th it started plummeting down to 300 a day. I have recently implemented this robots.txt file for controlling duplicate content on this wordpress blog. Am I somehow blocking other sites abilities to grab my feeds? Thanks

Actual file:

User-agent: *
Disallow:

Disallow: /wp-
Disallow: /search
Disallow: /feed
Disallow: /comments/feed
Disallow: /feed/$
Disallow: /*/feed/$
Disallow: /*/feed/rss/$
Disallow: /*/trackback/$
Disallow: /*/*/feed/$
Disallow: /*/*/feed/rss/$
Disallow: /*/*/trackback/$
Disallow: /*/*/*/feed/$
Disallow: /*/*/*/feed/rss/$
Disallow: /*/*/*/trackback/$

[edited by: Boulder90 at 7:13 pm (utc) on May 2, 2009]

[g1smd]

I would use only ONE single * in any one rule.

I'd imagine a parser choking on /*/*/*/

With simplification you only need six rules (or so).

[jbinbpt]

Did you run it through Google Webmaster tools Analyze Robots.txt?

[Boulder90]

Thanks g1smd. I'm getting the feeling that I am blocking feeds with my file which is why the traffic is way down. Are feeds really considered dupe content by Google?

jbin-

Yeah Google says it is fine.

[Boulder90]

G1smd -

How does this look?

Disallow: /wp-
Disallow: /search
Disallow: /feed
Disallow: /comments/feed
Disallow: /feed/$
Disallow: /*/feed/$
Disallow: /*/feed/rss/$
Disallow: /*/trackback/$
Disallow: /*/feed/$
Disallow: /*/feed/rss/$
Disallow: /*/trackback/$
Disallow: /*/feed/$
Disallow: /*/feed/rss/$
Disallow: /*/trackback/$

[g1smd]

You can delete the exact repetition of rules. You'll be left with about eight.

[Boulder90]

My traffic keeps plummeting, and google has droppd 70 pages from the non-supplemental and almost all of my images from the blog. Today I noticed that most of my traffic the last few days has come from some sort of russian blog with an emphasis on sex links. However, I could NOT find my link on that page even though Awstats say that's exactly where it's coming from.

What the heck is going on? There's no malicious code on my p[agesand I am certainly not placing my link on russion blogs with links to all things sex.

[londrum]

it might be one of those proxy hijacking things
try doing a search in google for inurl:example.com
where example.com is your domain name.

you want to look for loads of foreign pages where your url is part of the query string.

there's a long post on this site somewhere about how to stop it. can't remember exactly where, but a search for proxy hijacking should find it (if that's what it turns out to be)

[Boulder90]

Thanks for the comments, londrum. I checked that test and no luck. I have no idea what's going on. I've dropped 70% of my traffic, Google dropped my images and 70 pages of non-supps. The only thing I can point to is that the page responsible for most of my traffic in May is bizarre.

[Boulder90]

Alright, I just did a Google with the command:

site:example.com -inurl:www

and it shows about 30 pages of pron and warez links on my site at this URL:

http://example.com.com/examplefolder/wp-content/cache/userlogins/WP/warezand#*$!.html

I also notice a bunch of weird files in that folder on my FTP. Can I just delete all those files? All of the uer logins are people I don't know! Upon further comparison with my other wordpress files, it looks like this entire "cache" directory is completely bogus.

Is there a name for this kind of hack or spam? Whatever it's doing it's killed my bog since late April.

EDIT: after some snooping around my FTP, it looks like my site has been hacked and then somehow redirected. That explains over 100 pages being dropped from non-supps and all my images.

[edited by: Boulder90 at 4:25 am (utc) on May 10, 2009]

[g1smd]

Make sure all your software is up to date: Apache, PHP, FTP server, Wordpress, etc. Check your .htaccess and .htpasswd files for unauthorised modifications. Clear out all those new files, and change passwords, then speak to your host about the hack. It might be server wide, not just you.

[Boulder90]

Thanks g1smd.

Can bad sites linking to you have negative effects? I have two hobby blogs, and I notice that one of them has a bunch of spam pages linking to it. On my major hobby blog (the one referenced in this thread), I have *one* polish/spam site linking to it, but I cannot find my link on that page(even translated, etc)as the #1 referrer to my page. My non-supps are like ten pages now, they were well over 100, asnd the blog had images indexed in Google.

This stinks. If bad links can affect me, I don't know what to do. I can't control that.

[Boulder90]

An update to anyone bored enough to follow this pathetic story:

Turns out someone hacked my WP uploads folder and inserted this really weird XML.import file that redirected most of my blog to #$@# and pill sites. Google probably didn't like that, lol. They de-indexed all but 4 pages of my site and all it's images. I had well over 100 in the non-supps.

Poof! Good thing it's just a hobby/interest site. I would be even more upset if this was a commercial site that I relied on to pay the bills.

I've searched my blog for other malicious fiels but can't find any. Hopefully deleting that solves the problem.