Welcome to WebmasterWorld Guest from 54.225.56.78

Forum Moderators: Robert Charlton & aakk9999 & andy langton & goodroi

Message Too Old, No Replies

Indexed pages drop from 250,000 to 14,000

     
5:41 pm on May 2, 2009 (gmt 0)

New User

10+ Year Member

joined:Aug 15, 2005
posts:7
votes: 0


Hello,

we have a site with free, quality content. We launched it last summer, page indexing and ranking started well and were getting better until the end of september, when the traffic was around 30k unique per month. After that, our site dropped suddenly in rankings, and our traffic was around 0. We made no changes in that period and we do not use heavy SEO practises.

We do monitor the website with google webmaster tools and no problem is pointed out there.

In december the site appeared again in serp and our traffic rose a little, but was about 1/3 of september. We had another drop to zero in february and now we are back to december levels from march.
But from january to march we saw our indexed pages drop a lot, from about 250,000 pages to 14,000. Consider that our site has more than 1 million different pages.

What could we do to have our site not treated well, but at least decently.

Thanks in advance to all of you.

[edited by: tedster at 5:54 pm (utc) on May 2, 2009]

1:50 am on May 3, 2009 (gmt 0)

Senior Member

WebmasterWorld Senior Member tedster is a WebmasterWorld Top Contributor of All Time 10+ Year Member

joined:May 26, 2000
posts:37301
votes: 0


Some kind of ranking and traffic drop a month or two after launching a website is a common experience - but at this point it sounds like something is going on beyond that "honeymoon period" effect. It sounds like the site may not have very strong backlinks/PR - or there may be a technical problem involved, such as too many multiple urls for the same content, duplicate title elements or meta descriptions, server bugs, robots.txt errors - many technical possibilities come to mind.

Have you set up a Google Webmaster Tools account and checked in there for any reports or messages that might be involved in this indexing problem? I'd also suggest an XML sitemap if you aren't already generating one.

5:07 am on May 3, 2009 (gmt 0)

Senior Member

WebmasterWorld Senior Member 5+ Year Member

joined:Nov 3, 2006
posts:1337
votes: 1


I'd also suggest an XML sitemap if you aren't already generating one.
Tedster, do you update the XML sitemap or do most people have it programmed to do it automatically? Any links where I can learn more?
2:56 pm on May 3, 2009 (gmt 0)

Senior Member

WebmasterWorld Senior Member tedster is a WebmasterWorld Top Contributor of All Time 10+ Year Member

joined:May 26, 2000
posts:37301
votes: 0


Google has a nice full Help section about XML sitemaps:
[google.com...]

There are several approaches you can take, depending on your site's size and technical needs.

12:01 pm on May 4, 2009 (gmt 0)

New User

10+ Year Member

joined:Aug 15, 2005
posts:7
votes: 0


Thanks for your remarks.

We use webmaster tools and no problems are pointed out there.

Moreover, we have similar sites in other languages, and never had these kind of problems.

Another thing that is strange is googlebot activity graph, shown in webmaster tools. The number of pages scanned per day were high (max 43,000/day) and became low (min 813!) and they do not seem to come back higher. Don't know if there connection with the drop of indexed pages.

1:12 pm on May 5, 2009 (gmt 0)

Preferred Member

10+ Year Member

joined:Sept 2, 2003
posts:412
votes: 0


we have a site which is having almost 3000 pages with original content. since january 2008 we are trying to get them back in google but google is not indexing them at all. We are regularly adding fresh contnet pages in the site which are required but still there is no improvement. I tried deep search but google did not return enough pages.. although rankings are fine but still i believe if the pages will increase, i can get the more traffic.

its a old and good traffic site...any suggestion.

4:47 pm on May 5, 2009 (gmt 0)

Preferred Member

10+ Year Member

joined:Dec 30, 2002
posts:437
votes: 0


You have 250k pages and your site is less than a year old.

No specifics please, but where did you get the content for these 250k pages?

Is the content unique to the web?

How many of those 250k pages have incoming links from external sources? A better question might be what percentage? 1% of pages with incoming links would be 2500 pages. If only 1% of pages have incoming links, would you index 250k pages deep if you were google? No. (Maybe I'm off base here, but that's my quick thoughts given no context real of the situation.)

I suspect GoogleBot has determined, for either PageRank or quality reasons, that your site is not worthy of the level of crawl depth you would like to have.

To me, even 14k seems like a lot for a >1 year old site.

5:06 pm on May 5, 2009 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:Mar 6, 2002
posts:1831
votes: 22


Peter, I don't think so.

It does not matter how old your site is. Trusted links, PR, good content and not tripping any on-site penalties matters. 250k URLs indexed is not a problem at all for a new site.

6:26 pm on May 5, 2009 (gmt 0)

Preferred Member

10+ Year Member

joined:Dec 30, 2002
posts:437
votes: 0


Trusted links, PR, good content and not tripping any on-site penalties matters.

I don't disagree. I question whether all that is in place, and intended my remarks to dance around those topics.
11:14 am on Sept 2, 2009 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:Aug 17, 2004
posts:1354
votes: 0


If you have a 250k-page site with little inbound links, google might consider your content unworthy. Also, with that many apges, you have to make sure both the spider and any linkjuice can crawl freely throughout your site.
12:34 pm on Sept 2, 2009 (gmt 0)

Full Member

10+ Year Member

joined:Mar 31, 2005
posts:337
votes: 0


Consider that our site has more than 1 million different pages

in just over a year. How many people do you have working on the content for your site?
7:42 pm on Sept 2, 2009 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:Apr 9, 2005
posts:1509
votes: 0


Unless Google has changed the rules since I have been reading... It has been stated by Matt 'I' Cutts 'The Spam' 5000 pages a day is the max you should add to a site. It seems possible you could have steadily added 2700 (ish) pages a day to get to your current page count, but it also seems possible you tripped the 'too big, too fast' filter by adding too many pages at once...

One thing to keep in mind re this situation is if you 'added steadily' under the threshold, but Google didn't spider all 2700 new URLs one day (or even for a couple of days in a row) and the site is dynamic, which means it probably doesn't serve 'last modified' headers, when G came back and spidered again it could have looked like you added 5400 or 8100 (or more) pages in a single day, even though you really didn't...

 

Join The Conversation

Moderators and Top Contributors

Hot Threads This Week

Featured Threads

Free SEO Tools

Hire Expert Members