Rankings Improved When Deindexed Site Got Reindexed Whilst Blocked In robots.txt

[SerpsGuy]

Ill make this as simple as possible. I had a website in google, and I requested removal in webmaster tools. Site is completely removed in one day.

Then, 24 hours later, I request reinclusion, after blocking google via robots.txt. The idea being a test of the reinclusion tools speed. Within 4 hours this entire website is reindexed, but without titles, or meta description because googlebot cannot see the content.

The weird part? Almost all the links show up on page one. This domain is old, and has some good links built to it over time. But it has not ranked for a long time.

Seeing this, I decided to unblock robots.txt and see where this was headed. Immediately, after google crawled the content, I dropped back to pages 6-7-8.

Could this mean something? Maybe the content is bad? Or maybe this is a normal occurrence with google not being able to read your content?

[aakk9999]

On the surface it appears there are signals that your website content and its site structure are sending that caused the site to drop.

Interesting is that Google did not use historical data (your pages as it knew it) even though the re-inclusion was only 24 hours later on, and instead, as the site was blocked in robots.txt, it appeared to rank the website based on external signals only.

It would have been interesting to know whether the site would remain to rank well if it stayed blocked by robots.txt longer or would the drop happen anyway.

[SerpsGuy]

I did some searching of my own content, and there are hundreds of websites like this one:

<snip> The example was a low quality MFA site that appeared to have scraped SERPs including snippet

that copy my content. Its just a small snippet for every site like this one, but essentially most of my content exists on sites like this in small portions. Is there a way to prevent these junk sites from scraping my stuff?

[edited by: SerpsGuy at 8:35 pm (utc) on Oct 1, 2014]

[edited by: aakk9999 at 8:44 pm (utc) on Oct 1, 2014]
[edit reason] Sorry, no URLs that can end up with site review [/edit]

[Clay_More]

Interesting observation.

The weird part? Almost all the links show up on page one.

Where you searching terms/phrases the site previously ranked for when you saw the links show up on page one?

In your opinion, how competitive were the results for the terms you searched?

Thanks for any additional info you can share.

[SerpsGuy]

I think its semi competitive. Google shows 1.2 million to 2.5 million for most of the terms. The terms I searched to see my ranking was the keyword I optimized for. Not the URL or a quoted line or anything.

The websites I run are affiliate websites, but I focus mostly on in depth content covering every topic. And Im not talking crap like 'five reasons why you need a new hunting knife' kinda crap. In terms of competing affiliates, yes, its very competitive. Commissions are 35-45% so people tend to be willing to do pretty slimy things to take out a competitor.

[SerpsGuy]

aakk9999 - Your comment got me wondering. I am going to try the reinclude on friday, but keep google blocked via robots.txt and see how long the rankings hold.

If they hold - What tests do you suggest I try to learn something from this? Should I allow one url to be crawled and see if that affects the rest of them? What useful knowledge can be gained by testing this?

[Clay_More]

The first step is probably to determine if this is a "one off" kind of issue, or whether it can be duplicated with other sites.

I just deleted a website with about 1,400 backlinks and requested removal from Google for all it's pages. Tomorrow I'll block Google using robots.txt and shortly thereafter try to get the pages re-indexed. I'll leave the robots.txt in place for a while.

[SerpsGuy]

Cool - Hey could you share your results too? Id like to know if you have the same experience as me!

[Clay_More]

I could not duplicate your experience. My site disappeared and stayed that way.