Welcome to WebmasterWorld Guest from

Forum Moderators: open

Message Too Old, No Replies

More eBay, Amazon, Dealtime, Epinions, etc. in Results

Side Effects of Improved Dynamic Spidering?



3:21 pm on Aug 5, 2003 (gmt 0)

10+ Year Member

For a lot of the words I track, I've noticed a marked increase in the number of meta shopping sites: eBay, Amazon, Dealtime, Epinions, CNET Shopper, Buy.com and even Google Answers in the results.

Is this a side-effect of improved dynamic spidering, or have the web developers for the sites wised-up (en masse)?

Random forum posts and PDF's sometimes show up in the top 10 for words that are pricey at overture ($5+).

My hope is that this is just a temporary situation before the full update occurs.


5:31 pm on Aug 11, 2003 (gmt 0)

10+ Year Member

On my popular search term mentioned above I have just moved up to 10, back on first page.

Above me:

3 .gov sites that will be of no interest to searchers for this phrase.

A Stanford university paper that might be of some value to a small minority, but not really what they will be looking for.

A .pdf which won't be of real interest

Snapping at my heals, more .gov sites and 2 Amazon book sites.

Not only will the book sites be of no interest to anyone using this phrase, but one is for a year 2000 guide, just had a look and there are more up to date versions of the same book. hmmmmm.


5:53 pm on Aug 11, 2003 (gmt 0)

10+ Year Member

.PDFs and .DOCs are bad enough. But plain ole' message boards that have snuck up into the top 10 (3 message boards and 1 affiliate page not working to be exact)? What the....?


Keyword: Widgets
Message board URL: www.ukscoobydoo.co.uk/scoobydoodiving/forum/messages/1191.html

Then you click on the link and see:

Subject: 'Scooby Doo'

re: Go to my 'widgets' site and check out all the 'widgets' {link..link..etc.}

And 2 of these boards are totally different, but both have comments and links from the SAME person.

In the friggin top 10? What? Googleguy, please let us know when there will be some light shed on this. Thank you.


6:03 pm on Aug 11, 2003 (gmt 0)

10+ Year Member

SeventiesMartin, as I listed in my post above, 3 of the 10 results on page 1 were unrelated .GOV sites. That, combined with some other worthless results, show Google is having big problems with its algo changes.

Perhaps what we need is a better outlet for our frustrations? Webmaster World doesn't permit posting specific URLs. Googleguy may or may not be listening. Using Google's official spam report can be like a one-way black hole.

How about a independant 3rd party creating a new Google spam report website (www.google-spam.com is available). It appears just like Google's spam report, except that the results are posted for public viewing.

Fill in a spam report to Google. Send a duplicate report to Google-spam.com. Get it off your chest. Spammers will be embarrassed to see their domains listed. Google itself is free to peruse the site. Feature the top 10 examples of spam and irrelevant results. The possibilities are endless!

Silly idea? Probably. But this is the sort of desperate idea that come to mind after Google keeps pulling the rug from under your feet.


6:08 pm on Aug 11, 2003 (gmt 0)

10+ Year Member

Hey Umbra good idea, but what I'm seeing is mostly not spam. These are just regular pdfs, docs and ppts with some mention of the search phrase but with very little relevant content.


6:16 pm on Aug 11, 2003 (gmt 0)

10+ Year Member

Myself, I've been seeing a mix of both. In any case, reporting irrelevant results to this hypothetical website is just fine. A completely irrelevant site ranked #1 is even worse than a spammer occupying that niche.


6:26 pm on Aug 11, 2003 (gmt 0)

How about a independent 3rd party creating a new Google spam report website (www.google-spam.com is available). It appears just like Google's spam report, except that the results are posted for public viewing.

Good idea. But save yourself some grief and make it google-spam.org instead.

A small-time blogger had the scroogle.com domain, and got threatening emails from Google lawyer Rose Hagan complaining that "scroogle" was "phonetically similar" to "Google." I immediately registered scroogle.org and offered it to the beleaguered blogger for free if he wanted it.

The point is, I think you're safer with a dot-org than a dot-com in cases like this. It also helps to be clearly operating as a nonaffiliated nonprofit.


7:29 pm on Aug 11, 2003 (gmt 0)

I've been watching these forums for years, and I've never seen such open hostility towards Google - but last weeks events have seen extraordinary events.

There's no doubt that something very unusual has happened. But I suspect the 'Amazoogle' theory is probably wrong.

(By the way, I proposed the word Amazoogle, and was shot down in flames - but it's just a term that springs to mind when I look at the SERPS!)

What we may be seeing is an inflated version of what we used to call freshbot.

New crawl => new site that is initially high in the listings (this actually makes sense, if it is a good site folks need to see it for a few days so that it can be linked to!). Then it drops to find its proper place.

The Amazon sites etc. are more worrying - the affiliate system is so huge that without a tweak they'll ruin the serps.


7:33 pm on Aug 11, 2003 (gmt 0)

On the occassion that I have used the spam report I think it has worked out OK.

In fact I have done a report today for the Googlers to consider - note consider (Google might find the pages relevant to the search) - just making them aware of problems/unfair ranking.

However, the problems being discussed regarding PDF,Docs and Amazon pages is not a spam issue but an algo issue.

Cant say I have noticed Amazon,PDF or Doc problems myself ( unless the term is mentioned here ;) )but I have been working to much and surfing to little - wonder if anyone from Amazon has noticed this thread and if action was taken I am sure they would feel it was unfair - have they broken Google Spam guidelines?


7:39 pm on Aug 11, 2003 (gmt 0)

10+ Year Member

One thing that is interesting, is that usually in a thread discussing a perceived problem with google, there will be a pretty high percentage of posters defending google. This isn't the case so much in this thread, which may indicate that the percieved quality of the SERPS by members of webmasterworld are very low.


7:41 pm on Aug 11, 2003 (gmt 0)

10+ Year Member

Wow! I'm seeing Amazon as #1 and MSN search results as #3 and #4! This is for the search phrase "adidas golf shoes".

What the %$#%$# is going on! I hope this gets corrected soon!



8:06 pm on Aug 11, 2003 (gmt 0)

10+ Year Member

I haven't read absolutely all posts in this thread, so I apologize in advance if someone has already raised the question, but - how come this thread is still up and running? Isn't it totally against the (until now) rigidly enforced TOS of WebmasterWorld to not point at specific sites that do well in SEs? So what if these are big sites? What if I posted a thread whose subject is "how come www.Kackle.com is doing so well in Google?" Wouldn't that thread be (rightly) shot down before the ink is dry?
I think this thread is all wrong - either enforce the TOS all the way and remove this thread or cancel the TOS on not naming specific sites.


8:08 pm on Aug 11, 2003 (gmt 0)

Thanks Boaz, for giving WebmasterWorld an excuse to delete the entire thread.

Well done!

See you all in cyberspace!


8:26 pm on Aug 11, 2003 (gmt 0)

10+ Year Member

"Well done!" - that is my intention - getting it deleted. While you may not appreciate it, today it is these sites that are at the spotlight, and tomorrow it is your site that is the topic. Would you like that? I don't think so. There's a reason for this TOS.


8:53 pm on Aug 11, 2003 (gmt 0)

10+ Year Member

Everyone does stick to the TOS when it comes to the usual spammer type sites.

Here we are talking about a site with nearly 8 million pages (allinurl:) dominating the SERPS in most search areas (and it would seem with results that are mainly not relevant) to the detriment of the user experience.

If we can't discuss that then there's not much point in a forum.

Also, the pdf's, and docs that are being talked about are not being mentioned by site name, just document type.


9:15 pm on Aug 11, 2003 (gmt 0)

10+ Year Member

Regarding the PDFs and DOCs, you're right. As to the rest I disagree - there were various sites pointed at (4 just in the title and more if you read the various posts), not just one.

"If we can't discuss that then there's not much point in a forum." - I disagree, WebmasterWorld does well even with this TOS strictly enforced. I agree it's annoying sometimes, but it's there for a reason, and not just to protect spammer sites....


9:22 pm on Aug 11, 2003 (gmt 0)

10+ Year Member

Boaz - This forum is where the rubber meets the road. This is where feedback from the webmaster community can be heard, responded to and debated. This is also where someone from Google has the chance to hear us. If one of us should see something we believe is highly detrimental, like another SE's results pages being ranked high in the Google index, this is where to point it out.

When a major company, or group of companies, can come into the Google index and absolutely dominate thousands of markets, that's a problem. That's a problem for the end user, that's a problem for webmasters and ultimately that's a problem for Google, as joe user will simply switch search engines in search of relevent results.

I won't go as far as to say that Google's days are numbered. However, there is a lot of negative energy swirling around about Google right now. Not just from webmasters but end users as well.

I work in the high tech industry. Just a short time ago you were considered a dinosaur if you didn't use Google as your SE. That is no longer the case. Now you are "quaint", and in danger of being "lame" if you use Google. That's not a good sign.

If Google can't regain some semblance of stability and relevence they have a tough time ahead.

Also, I would never advocate using someone's domain name here. Unfortunately the companies in question here are so big that they are literally household names. Shall I say "That really big online book company that also has web stores and other products as well"? Or perhaps "That other big SE that Microsoft owns"?


9:56 pm on Aug 11, 2003 (gmt 0)

10+ Year Member

Amazon moved above us on a lot of results, one of our competitors is an Amazon merchant, so now they are getting more play.


10:03 pm on Aug 11, 2003 (gmt 0)

10+ Year Member

It should also be said that this thread is not about knocking google for the sake of it.

In many areas the SERPS aren't up to the quality that we know they can be at the moment.

This forum is our only way to discuss what we see. It is also a way we can give feedback as GG reads this forum. GG takes note of what's mentioned here, and I'm sure google get to know of problems sooner than they might have done because of Webmasterworld.

In cases like this where extemely large sites are completely dominating search results we can't discuss and give that feedback without mentioning names.


10:04 pm on Aug 11, 2003 (gmt 0)

Ladies and Gentlemen,

The solution is simple:

Buy the full version of Acrobat and upload all your pages into this format - and convert to Word too. Stick your site-map in it. Excel won't do any harm either!

Publish your site in as many languages as possible; become a reseller of everything you can think of. Become an Amazon associate.

But most cunning of all - write a BOOK about your site, and publish it, and ensure that Amazon sell it.


10:05 pm on Aug 11, 2003 (gmt 0)

10+ Year Member

Hmmm...Not sure if I can entirely agree here with the demise of Google or the poor quality searches. While I will agree that there ARE some things that Google needs to look at, the simple fact is I believe their SE results are still the best.

First, to compare Ink to Google is almost funny. The actual results by Ink aren't awful, but hardly compare to Google. The reason? Ink has nothing in their database! I did a search on pure search for various domain names, to see how many pages were listed in their index. Google has virtually all of mine (over 2000), but only 160 in Pure Search. A friends website is almost identical. Even larger sites, though, suffer the same fate. I checked a number of major catalog merchants domains and found the same thing - only a small fraction of their pages are listed in the Ink database. Thus, Ink has no where near the index that Google does, which limits it's usefulness.

Second, for many searches, there is nothing wrong with Amazon appearing. Many of the SE results are dead on for what Amazon has. While it isn't perfect, for sure (I too have seen some Amazon results come up that leave you scratching your head), for the most part, Amazon generally deserves the rankings it gets.

Third, completely agree about Kelkoo and Netster. They are listed everywhere in the SERPS and add absolutely nothing to the quality of the SERPS.

Fourth, another big question which is very justified, why are all these other search engine results appearing in Google Listings. I mean, this is probably the biggest spam of all - having other sites somehow (I have no idea how) manipulate Ms. Googlebot into ranking their site for virtually everykind of keyword combination available. And, as previously posted, most of these pages lead to 404's by the time they appear in Google's index.

Fifth, while I'm not a fan of DealTime, it's a popular site and it works well enough for what it does. I can understand why it is ranked as highly as it is. However, is there really a need to rank all the "flavors" of dealtime?

Sixth, while I've seen more PDF and DOC files in the SE's (which can be irritating, especially DOC files). However, many of these PDF's HAVE been on target - frequently deserving to be ranked where they are.

Ultimately, Google is always going to battle spam. The majority of the "old spam" sites seem to have been flushed from their system, with some newer sites taking their place. As always, Google will have to tweak their algo to get rid of these sites to keep their rankings ok.

Right now, from where I sit, I see two big sources of SE clutter that Google needs to guard against. First, they need to quit listing other Search Engine Results. Why search the results of another search engine? I went to Google to find something - not to be handed off to another Search Engine.

Secondly, with the rise of Google AdSense, has also arisen quite a crop of absolutely useless sites designed for one thing and one thing only - sell Google AdSense clicks. These sites are pure spam (consist of nothing but links to nearly carbon copy pages, a Google AdSense Ad, and a very long, plainly visible thing of keyword text), but, because they don't do anything "dirty" (no hidden text, doorway pages, etc...), they get picked up by Google and often rank quite highly.

Anyways, just my thoughts.



10:11 pm on Aug 11, 2003 (gmt 0)

WebmasterWorld Senior Member steveb is a WebmasterWorld Top Contributor of All Time 10+ Year Member

"But Google can't just tweak the algo because the whole ranking system is based on Pagerank."

This is totally untrue, and its bizarre to even see it mentioned.

The problems with the serps are a LACK of pagerank. The serps are ALL about anchor text.

Do an allinanchor search for a term. Do a regular search for a term. The results are normally nearly identical. Jeez, its plain as day if you spend five seconds checking.

This is all about Google's obsession with anchor text and its huge overraction to the selling of pagerank by drastically slashing the value of pagerank in the algorithm.


10:14 pm on Aug 11, 2003 (gmt 0)


Can this explain the Amazon deluge?


10:22 pm on Aug 11, 2003 (gmt 0)

10+ Year Member


I agree with several of your points. However, what is at question here is the future of Google search results. If Amazon can dominate the serps in this way, and I can't agree that the pages I've seen deserve to be there, then what's next? Do other huge companies simply list a billion items from a database and domintate as well? What happens when 5 or more big companies, all selling the same things, dominate the top 10 or 15 listings on Google?

I think that companies like Amazon should be listed in the serps. I would expect a search like "online book dealers" to have Amazon in the top 10, perhaps #1, instead Amazon isn't even there.

However, I would not expect a search term like "chiltons repair manual" to have Amazon in the #1 position, as they are now. I would expect the chiltons company to be above Amazon in the serps on their own product.

The bottom line is that no one wants to see a bizillion Amazon pages in the serps.


10:43 pm on Aug 11, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member

>> "no one wants to see a bizillion Amazon pages in the serps."

And if they do, there's another online source for them, it's called "Amazon Search" on Amazon's homepage....


10:54 pm on Aug 11, 2003 (gmt 0)

10+ Year Member

Hi c1bernaught,

Using the search term you provided, I agree Amazon probably shouldn't be the number 1 slot - but it should most definitely be in the top 10, if not number 2. After all, the search term you used is for a "book" - a title of a book, in fact. And, in regards to books, Amazon is likely to be number 1 for many reasons.

And the reason I suspect Chiltons own website is down at number 5 (behind Yahoo Auctions and a Auto Gear Site) is due to so-so SE technique and a lack of back-links. Some more backlinks and removing the frameset would probably help increase the rankings of Chiltons Site.



12:06 am on Aug 12, 2003 (gmt 0)

10+ Year Member

Hey Guys,


Ill pay $1,000 to anybody can tell me how Amazon gets a #7 and #8 ranking on the word "car parts" ... it doesnt even mention the word.

This is a VERY hard word to get ranked for, all other sites in the top 10 have over 60-200 links.

Im not picking on amazon but look at the page.


12:27 am on Aug 12, 2003 (gmt 0)

10+ Year Member

"Car Parts" is in the Page Title and I assume that it's mention in anchor text as well.


12:49 am on Aug 12, 2003 (gmt 0)

WebmasterWorld Senior Member googleguy is a WebmasterWorld Top Contributor of All Time 10+ Year Member

Hey all, I'm still digging. I've gotten about eight pdf reports. About five of them are from one fellow in a more technical field where I'm not amazed that pdfs would show up, but I'm still trying to investigate more on them all. Investigating reports of more commercial/vendor pages is harder than pdf, but I'm seeing what I find there too. I haven't come up with any clinchers so far, although I'm still trying. Special thanks to the member from here who sent a nice note about this thread and their thoughts on it (they know who they are).


12:58 am on Aug 12, 2003 (gmt 0)

10+ Year Member

site:amazon.com "car+parts" : 2,840,000
site:amazon.com "books car parts" : 1 result
site:amazon.com "book car parts" : 0 result
+the site:amazon.com : 8,390,000

Amazon Search
Books: See all 106 results...
Baby: See all 108 results... (mainly baby seat for car)
No comments!


1:37 am on Aug 12, 2003 (gmt 0)

10+ Year Member

GG "About five of them are from one fellow in a more technical field where I'm not amazed that pdfs would show up, but I'm still trying to investigate more on them all."

I assume you're referring to me, and I'm sorry but three of those were the broadest search terms possible with 1.7 million, 2.9 million and 4.5 million results. Data storage is by no means a technical search. Of those three, two pertained to my site. The other two did not pertain to my site, and they were picked at random. PDFs have never appeared in the top 30 results for the two that I monitor closely. If that is the case, I could put up 20 - 30 page detailed pdfs all day long to compete. As I've mentioned before, I feel that a pdf takes away from the users experience unless it is a very technical search, otherwise I'd provide more for my users. But even if you take all of that out of the picture, I'm talking about pdfs/docs that have little relevant content.

I appreciate you looking into it, and if you'd like for me to explain further or send more examples then I will.

One final note, I've searched in areas where I do have pdfs, and I've found them near the top of the results. They don't deserve to be there either.

This 274 message thread spans 10 pages: 274

Featured Threads

Hot Threads This Week

Hot Threads This Month