Google removing thousands of pages

[skuba]

Hi,
I have some Amazon affiliated websites running a script based on Amazon Web Services, Amazon Products Feed.

For the past couple weeks Google had removed thousands of the pages from their listing. Among the pages that remain, most of them only have an URL, they don't have description or title anymore.

One of my sites used to have 80k pages listed, now it's down to 25k and most page without any title or description. Traffic to this site went from 200+ users a day to 10.

I would like to know if it's a known fact that Google in on an effort to remove sites running Amazon Products Feed?

I don't see other sites using scripts based on AWS been removed from google. I also don't see sites running other scripts for other affiliate programs getting removed. It looks like there is an specific attack on Amazon Products Feed sites.

Any thoughts about that? Thanks a lot for your help.

[edited by: Marcia at 2:28 am (utc) on May 25, 2004]
[edit reason] No specific sites, please. [/edit]

[Marcia]

There have been a number of people experiencing pages being removed from large sites.

It's entirely possible that we might be seeing more of an effort to eliminate near-duplicate pages.

[jbgilbert]

It's not just Amazon feeds! Appears to be large sites where pages have "too much" in common.

[anallawalla]

Suits my client fine - I saw a strange site where each Affiliate has a PPC landing page that's a copy of the index page, e.g. 12345.html, 12346.html etc. Every page on this site shows just the URL in the index. The dupe filter must be working.

Not a whisper of this in the APF support forum. If it spreads, a lot of affs will be concerned.

[quotations]

One of my sites used to have 80k pages listed, now it's down to 25k and most page without any title or description. Traffic to this site went from 200+ users a day to 10.

Any thoughts about that?

You have a site with over 80K pages and it only gets 200+ users per day?

Did I read that right?

That is less than one visitor per page per year.

Why is that worthwhile?

I have a hard time worrying about the individual pages which get less than a couple of hundred visitors per day. An 80,000 page site should be getting hundreds of thousands of visitors per day.

[jcoronella]

Some light reading for you:

Big sites suffering no title / no snippet in SERPS [webmasterworld.com]

Likely a few amazon aff's posting there - but it's not just amazon aff's that this has been affecting.

[shrirch]

Few questions for you

1) Have you got deep, non reciprocal inbounds?
2) Are you using an unmodified script / template which ten thousand if not more associates are using?

[ogletree]

I know of a site that has pages in the thousands and the number in Google keeps going up. It has fluctuated a little up and down but over a month the trend is up. You can not have 80K of pages that are at least 15% different unless you put a lot of effort into it. It can be done you just need to know how to do it. Look at the sites that are still in and I bet they are not doing the same thing you are.

[skuba]

Quotations

You have a site with over 80K pages and it only gets 200+ users per day?

Did I read that right?

That is less than one visitor per page per year.

Why is that worthwhile

No you don't get hundreds of thousands a day becuase you have many competing sites AND becuase there are not hundreds of thousands searches a day for all the products sold on the site. You probanly get hundreds of thousands in a month.

Shrirch

1) Have you got deep, non reciprocal inbounds?
2) Are you using an unmodified script / template which ten thousand if not more associates are using?

Some links, and I did modify some things on the script. But, yes, a lot of people use the same script.

[kenfused]

Same with my sites.
One of my sites had 60K pages indexed, now down about 25% over last maybe 6-8 weeks...

Maybe there is a way to redesign the URL?

[skuba]

That's what I am trying to find out. What triggers the filter and what could be done to resuscitate the website.

[ogletree]

We have seen that google seems to throttle new pages to about 2500. After that it is much slower to add. I think google is starting to shifting some attention to sites with lots of pages. It might be better to have several smaller sites with 2500 pages each.

[Marcia]

I'd hazard a wild guess that they've begun looking for certain strategic trigger points on pages that indicate the possibility of duplicate or near-duplicate content.

It can't be too easy on a mass scale, but I hear they've got about 60 or so people there who went to school and are almost as smart as SEOs.

[claus]

>> One of my sites used to have 80k pages listed, now it's down to 25k

I've been tracking this for a while with the G Query: "bbc site:bbc.co.uk" (*)

• Oct 12, 2003 [webmasterworld.com]: 3,100,000 pages (msg #6)
  
• Apr 09, 2004 [webmasterworld.com]: 1,350,000 pages (msg #5)
  
• May 26, 2004: 660,000 pages
  

- so, basically, you're not the only one, and it's not something new. Not commercial either, just plain size.

(*) As non-competitive and non-commercial as queries can be.

[macrost]

If your urls have asin in it, then take that out, replace it, or whatever. I can usually spot an amazon affiliate by that right there.

[Chico_Loco]

Might I ask what the PR of the site WAS & IS?

Given that the BBC are having the same issue - I'm not thinking it's related, but it might help.

Also - I am not sure how on earth you have only 200+ users with 80k pages. I have a smaller site with about 3k pages, and it gets at least 1k uniques per day, and this is in a pretty non-competitive industry. It's a hobby electronics site and certainly not a site your average user would look at, or even understand.

[renee]

>>If your urls have asin in it ...

macrost,

do you have any proof of this?

[oaktown]

It's happened to me also Skuba. One of my sites has 200k+ pages and over 4K backlinks. It's bad enough that G only shows less than 250 backlinks right now, but the vast majority of the pages (90%+) have gone missing. On April 2 our daily traffic dropped from an average of 30K visitors per day to next to nothing. We have no connection at all to Amazon, so I suspect that it is in fact a filter that demands a certain level of difference between pages or else it blows them away.

[skuba]

It's happened to me also Skuba. One of my sites has 200k+ pages and over 4K backlinks. It's bad enough that G only shows less than 250 backlinks right now, but the vast majority of the pages (90%+) have gone missing. On April 2 our daily traffic dropped from an average of 30K visitors per day to next to nothing. We have no connection at all to Amazon, so I suspect that it is in fact a filter that demands a certain level of difference between pages or else it blows them away.

But this is one of the thing I wonder. It needs to have a difference between our content and some other site content or a difference between our pages?

If the second is true, it's not very fair considering that most sites (specially ecommerce) have very similar pages, just changing the products offered in each page. This is actually true to even content sites.

[walkman]

I know it sucks for you and I would feel the same but look from Google's perspective; they aren't losing anything. Those products will still be shown via amazon's pages or hundreds of other sites running amazon store scripts.

[ogletree]

It's not that hard just put a little effort into creating different pages. We do it programaticly. Let me know if you need any help.

[skuba]

The pages are different. Most of the text is different. In each page you have different menus, links, products and text. But the layout is still the same, so you have the same tags <table> <td> etc....
The site is dynamic, you can't be changing the design for every single page. Where usability goes then?

I think what is considered duplicate content nowadays, it if one site is too similar ith another site that ranks higher.

[ThomasB]

We have seen that google seems to throttle new pages to about 2500

nope, 15k within 1 week ... still as it always used to be, fortunately.

[ogletree]

Do you add pages to that site like that all the time. Did your site have very many pages before you did that? Were the pages generated prgramaticly? How different were the pages? What was your linking structure? Do you get deep links to your different site map pages.

[ThomasB]

new domain, db driven, hierachy structure, no external deeplinks I'm aware of, similiar pages

Bit in a hurry, sorry for the short answer

[claus]

I should add that you shouldn't really think of the results count as accurate, if it's above (guess) 1,000 it's more like an estimate.

Google may show more pages or less pages than the number that actually exist in the index. See msg # 14 here: [webmasterworld.com...]