Forum Moderators: Robert Charlton & goodroi

Message Too Old, No Replies

Time needed to re-index a mega site? your opinion

         

cooler29

5:14 pm on Jan 29, 2011 (gmt 0)

10+ Year Member Top Contributors Of The Month



I started year and half ago new project(site) which was combination of user generated and aggregated content. User generated content is unique, mostly long posts(articles) and aggregated part is from careful selected sourced, very well structured, and with some other additional value.

When I started that site, my idea was only to experiment, to try/test implementing few ideas that I had without any further ideas what I will do with it, just to use it also in future to try some new stuffs, mostly related to testing some semantical and seo stuffs.

In mean time, after first year and getting some decent traffic, I found out that would be a shame not to use that site for profits, so I work few months,changed a 'direction' in terms to be more contributors oriented,get some new members, and after that started with ads. Traffic, and ads params (eCPM, eCTR, bounce rates, etc) were just great. After month and half, traffic in one day dropped to almost nothing (x.000% down:) ) In all that time, traffic was always better 50% than previous month, until totally drop, and that happens in beginning of September 2010.

Now some facts before drop: all that time number of pages in g index(using site operator) increased up to 200.000 pages (site have about 3.000.000), for first year about 25.000 pages received one or more visits, before drop G crawled about 10.000 pages daily, I had some back links from reputable sites (not paid reviews, people just made posts and discussions about site, few of them are well known blogs) and a lot sites that podcasting my original and aggregated content (aggregated but with additional value) with or without backlink to my site. In webmasters tools I had about 9.000 pages were 100% internal duplicated content ('small' mistake in htaccess) but due to my starting plans with this site I was to lazy to fix that. About 90% was www and non-www duplicated content problem and about 10% was problem that same page will came up under slightly different URL then real one.

Now some facts after traffic drop:site traffic drop to almost nothing like I said, so few days after drop, I fixed all internal duplicated content problems, improved internal structure, done much more on site quality (which I really think was decent even before drop) and implemented some new ideas related to semantics and relevance in internal structure. After few days, at the end of first week of Sep 2010, google start crawling my site at rate better than ever about 50-60K daily. And 2 weeks ago (according to this moment) crawl rate dropped to stable 5K daily.Traffic from G is still almost nothing, but yahoo and bing in past few months gives some decent traffic. But that is not a point, I wan't G traffic :) Or at least I want to know what problem is. Using site operator shows less than 100 results. At site have rank number 1 for own name :) All sites that podcast my content are better ranked and my site in most cases isn't ranked at all on my own original and unique content.

It is not my priority to back this site on level before drop, my main goal in this moment is to try to understand what happens there. Never had similar experience so I would like to share your opinion according to your experiences about this:

QUESTION:Is it possible that G just re indexing my fixed content (www->non-www)? And if it is, what do you think what time it need to finish that?



To made stuffs more interesting, in Dec 2010, when I figured out that this need to take some time, decided to do one more experiment.
Purcshased few new (niche) domains, and started to steal content from myself :)
Ok it is not actually stealing, but here what that is:
few niche domains, and api that allow niche site to podcast main site content but with some more additional value than other who syndicating my content using my main site rss feeds, categories and better descriptions. So every niche taking content in its niche not all content from main site. Those few aggregators that i made to podcast main site content, and are in this manner: part of content and appropriate category on main site, better description that every other aggregator who use my feeds, and no back links to my main site. And every of them was very good structured. From Dec 2010 to this moment all sites have great G traffic, increasing weekly for 10-15%. Forgot to say, that is with very few back links to each of this sites when they are published and after that I din't even tried to build some links.
I don't need that sites, because they are like I said, just for experimenting, but why G loves them more than main site? Or that is just looks from this moment perspective?


QUESTION2:Do you think that I should drop my 'main' site that I wrote about and continue to use it just for experimenting or I should fight and invest my time to solve problems and develop my main site into profit site, with solid number of contributors(members) that generate quality content?

Motivation to post, was last few days discussion about what are content farms and how G will treat them in future, so I would appreciate your opinion about this 2 questions.

tedster

7:14 pm on Jan 29, 2011 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



Yes, it's very possible that Google just finished - or maybe they are not even finished. Assessing a lot of canonical changes, especially when 301 redirects are involved, can be a very slow process even if all the changes are now technically sound.

On your second question, it sounds like this is the kind of site that Google might be aiming at with their recent algo change. The newer, smaller sites may well look more tightly focused to Google's "document classifier" system.

But it's hard to say anything definitive. I'm pretty sure this is just step one in an evolving whirlwind of algorithm shifts.

cooler29

8:12 pm on Jan 29, 2011 (gmt 0)

10+ Year Member Top Contributors Of The Month



I know that is hard to say anything definitive. I don't expect 'solution' just want to see what other thinks, so thanks for your opinion about both of mine questions.

I would like to see what other members think to...

tedster

8:20 pm on Jan 29, 2011 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



I would too. You shared a lot of good detail.

And also - welcome to the forums. I'm happy to see another person at the party.

Some added detail. I know of a large site that did canonical 301 redirects from no-www to with-www. Before they did the redirects, they had an almost equal number of URLs from each type. Four months later their rankings were still not recovered. At the same time, they had a few other technical mis-steps along the way, such as leaving in-content links that pointed to the no-www version.

cooler29

1:05 pm on Jan 30, 2011 (gmt 0)

10+ Year Member Top Contributors Of The Month



Thanks for welcome tedster. And I am sorry that didn't become part of this community earlier.

Also, thanks for this info about another mega site that you mention, this gives me some clues and parts of puzzle.

cooler29

5:15 pm on Feb 12, 2011 (gmt 0)

10+ Year Member Top Contributors Of The Month



I also forgot to add 1 more info: in last to 2 or even 3 months, whenever something new is published on site (short or long article, unique or pulled from rss) google bot access in 15-120 seconds from moment of publishing, maximum 2 minutes until google bot access new content for first time, but I don't see any of those pages in index. This is accurate measure since I made script that takes care about this. Any new opinions?