Forum Moderators: Robert Charlton & goodroi
I would like to see if we can break down the reasons why we dropped out of the SERPS, and offer ways to protect our domains in the future. I want to see how many of us have something in common: (more statements can be added)
1. Are you in a competitive money area? (yeah but keywords are not too bad)
2. Are you an adsense publisher? (yes, but really, really doubt this has anything to do with it)
3. Did you make alot of money from Google traffic? (nah, not yet..but was growing enough where it would stink if I lost it...I lost it)
4. Dedicated or Shared IP? Do you think switching to a dedicated IP from shared will improve rankings? (shared)
5. Do you believe you were penalized? If you put in keywords that only you should show up as, are other sites outranking you? (definitely believe I was penalized)
6. Did you forget the 301 redirect, to redirect non-www to www vice versa? (I did!) Are there any other htaccess codes to put in by the way?
7. For those of you in the sandbox before Allegra, did Allegra take you out of the sandbox, then did it dump you back in the sandbox in the Bourbon update? (Yes, I was out of the sanbox due to the allegra update, now im even worse than I was 9 months ago...cant even rank for a phrase with 1000 results. Its to the point where I can only rank if NO website contains the keyword.)
8. How many keywords per webpage do you use when optimizing? How many words per page on average? (about 6-10 keywords...about 180 words per page, keyword density is 10/180 = about 5% max most of the time...sometimes maybe 10% if I have a short writeup)
9. Proximity of keywords? Are they too close? (My headers are always [h1]keyword[/h1], and the keyword also starts at the beginning of my first sentence usually...so the header keyword, and regular keyword are adjacent. Other than that, keywords for me are naturally spaced apart for the most part...about 9-15+ words in between the keywords.)
10. How old is your site? (10 months)
11. How did you obtain links when you first started out? How often do you get new quality links? Did you stop obtaining links, thinking you would continue to get them naturally? (I politely asked other music site webmasters if they would like to exchange links. I stopped doing this after I ranked in G, because I realized I needed more content.)
12. Do spammy sites outrank you? (yes!)
13. Do you do well in Yahoo, MSN, etc? (yes!)
14. Do you hate Google. (no I think their search engine is great, despite my horrible rankings)
Well, maybe we can all communicate to each other what went wrong. If we find out that there is enough similarities between certain things, maybe that could be one cause of our dropped rankings. Perhaps everyone has number 4 and 6 in common. This may mean there is a relationship between 301 redirects, and dedicated ip's. Please, set your anger aside for the moment and lets just talk about the possibilities. Be nice too. No bashing Google, it will defeat the purpose of this thread. Thank you.
Somewhat.
2. Are you an adsense publisher?
Yes
3. Did you make alot of money from Google traffic?
60-70% of my traffic was coing from Google, so I took a big hit.
4. Dedicated or Shared IP? Do you think switching to a dedicated IP from shared will improve rankings?
Shared.
5. Do you believe you were penalized? If you put in keywords that only you should show up as, are other sites outranking you?
I don't know if I was penalized, but if I was, it was probably because I overused keywords and/or some of my pages are less than 150 words of text (but that's the nature of my particular niche.)
6. Did you forget the 301 redirect, to redirect non-www to www vice versa? Are there any other htaccess codes to put in by the way?
Have never used one and don't think it was ever a problem.
7. For those of you in the sandbox before Allegra, did Allegra take you out of the sandbox, then did it dump you back in the sandbox in the Bourbon update?
Way out of sandbox before allegra.
8. How many keywords per webpage do you use when optimizing? How many words per page on average?
I use maybe 20-30 keywords per page. Maybe that's my problem, because I rank well on Yahoo for some searches, not so good on others. Mostly bad on G, for everything now, especially pages which have the exact keywords in the title.
9. Proximity of keywords? Are they too close?
Keywords are close to the top, usually bold, size 5, not H1.
10. How old is your site?
6 years old.
11. How did you obtain links when you first started out? How often do you get new quality links? Did you stop obtaining links, thinking you would continue to get them naturally?
I never really pursued links because I was busy creating content, and I got quality links because of that. I have links in articles 5 years old that still show up.
12. Do spammy sites outrank you?
Sometimes yes, sometimes not, depends on the search.
13. Do you do well in Yahoo, MSN, etc?
Depends on the keywords.
14. Do you hate Google.
Hate is not the right word for me. I think they are misguided, mostly, as are most big, publicly-owned internet companies.
Looks like nearly everyone dumped had Adsense in common. Never had any problem ranking on G in almost 10 yrs, before we added Adsense. Our "site" is actually composed of about 3-4 domains interlinked. Only 1 was affected... the one most popular and the most heavily Adsensed. The others were all completely, untouched.
I hate to say it, but the argument that adsense publishers were penalized to some degree still has legs.
The way I see Google now is that of a very inconsistent SE with lots of conflict (Adwords, Adsense, GMail) getting in the way of being a true QUALITY SE.
This may only be a coincidence, but the few pages on my site that rank slightly better my index page (which has all the inbound links) are the pages WITHOUT AdSense units.
(1) Most posters in this forum use Adsense;
(2) Most sites created to display Adsense are of extremely low value to surfers;
(3) Google has additional feedback available from Adsense sites; their new algorithm correlates this feedback against other site statistics to spot patterns of "artificial site boosting" that weren't automatically visible before
(4) Google is blindly denigrating Adsense users.
The evidence cited here is obviously not so strong as to REQUIRE faith in MORE than one of these. I firmly believe #2, but still find #1 and #3 extremely credible. I'm not wearing a tinfoil hat and am therefore unable to focus on the other number, whatever it was.
(2) Most sites created to display Adsense are of extremely low value to surfers;
If Google thought this, wouldn't they ban the adsense account instead of keeping the account active and penalizing the site from their SE? I still make money from Yahoo and MSN traffic and my adsense account is not banned. If Google had a problem with their adsense being on my site, they would have either notified me, or banned my account.
(3) Google has additional feedback available from Adsense sites; their new algorithm correlates this feedback against other site statistics to spot patterns of "artificial site boosting" that weren't automatically visible before
If that is true, that algorithm would be seriously flawed because there are many innocents who were affected, and many guilty sites that still do well.
I don't believe adsense has anything to do with rankings, although anything is still possible. I think it is interesting as well though, that many of us are publishers. If there was in fact, any correlation, I would remove adsense from my whole website, as my site is not a "made for adsense" site. It is just a way to make some extra money while I build the site, but the money from adsense is not worth losing rankings from their engine. I did plan my site to be a keeper, and I don't know why it would be considered spam, as maybe their "Trust Rank" may suggest.
[edited by: chopin2256 at 5:57 pm (utc) on May 30, 2005]
It was less than two weeks ago that I started my interest in this topic of search engines, listings, etc, as my job in R&D was outsourced to Asia. Since I have only been at this for such a short time, I am reluctant to post what I have found so far, but it does seem relevant. Perhaps someone can offer guidance from what I have found so far and offer suggestions for improvement. My goal is to first understand search engines, and then to start a web based business.
Overall it appears to me from my research so far that Google is attempting to slowly educate their consumers to the same type of thinking that the phone company once did with the Yellow Pages. Consider that over many years consumers were actually taught to look in one directory when they are spending money and another when they are either not spending money, or have already decided where to spend it. It became natural to look "in the Yellow Pages." In Googles case, they want it to become natural to select from the right side of the page when you are spending money (and the very top) and from the left side when you aren't.
This is probably a five year or more process of consumer education and will require incremental changes. As Google reaches target for each incrimental step, they will likely change their algorithm to be closer to this ideal, where every commercial site pays to be listed on the right hand side. How incrimental each change is going to be I don't know. I am sure the Google toolbar is of significant help to their understanding of when it is time for the next change.
Also, as part of my research I built my own search engine and experimented with that the last couple of days. What I found there is that it really is possible to see quite a lot about a site by their keywords and by their links. If you search a large site and go five links deep, some of them will mostly stay on topic, and some will be all ads.
One site I searched had 20,800 links going five deep and almost all of them were either their own site or ads, or commercial sites. Others quickly linked out to many sites similar to themselves, and they in turn linked out to still other sites also similar.
Experimentation of reading the robots.txt and following the rules versus not reading them and just crawling resulted in signficant differences. These differences were so significant that I cannot imagine that a search engine could be relative if they always announced their presence and follow the robot.txt rules. I feel certain I can say that no reliable search engine would only follow the rules when establishing relevance.
Anyway, my point is that it is really possible to still tell something about a website not only by where it links to, but where the sites that it links to, links to, for several deep. Before I linked to a site, I would run a search engine on them three or four links deep to see where they go.
It is difficult to explain just how much can be seen about a page by looking at the five or six deep linking, but there is a lot of information there.
As a novice search engine builder there are some things that are easily evident. For example, it seems completely obvious to me at the novice level that one would want to gauge the click through rate of a page, and use that information for rating as well. So, for the person who said that they used to turn up number one on a rather generic term and now are buried, I would suggest it is because the text that the search engine returns for this generic search doesn't inspire a click through.
Other rather obvious things that I would look for are in the "bang for the buck" catagory. (Since I am just starting out, I am wanting a large return for a small effort.) For example, IP addresses can be correlated to location. If someone has a IP address that appears to be in CA and their IP address is also in CA and there is also a phone number that has a CA area code near the word "Phone" then that would have a heavy meaning.
If I had the luxury of having people search on my crawler, I would likely want to include a cookie that determined how long they actually stayed on someone's site. If they consistently return three seconds later, then I would know how to rank that sites relevance for the particular search term.
Not sure what else to say except that when I see talk that Google is ignoring the link information, I find it hard to believe. It seems to me that I can tell as much or more about a site by where it links, and where those links link to, than I can by the key words. Also the types words that are not keywords also seem important. For example, one rather conservative news outlet has signficantly higher usages of certain charged words like, "Patriotic, insurgents" than other news sites. I am sure that people at Google are probably intrigued by this sort of charged word, just as I am, though I haven't figured out any good bang for the buck stuff, they probably have.
Oh, in my few short days I have already come to disdain links with the word "ad" in them. I am even thinking about ignoring them altogether and just giving a downgrade for them. I am not suggesting that Google does this, only that I am finding that the text in the links themselves have some relevance as well, and that is the entire search engine thing, to establish relevance. When I get my site going more than sandbox, I will definitely pay attention to the actual names of the links.
If anyone has any good bang for the buck stuff I can use, it would be appreciated. I am intrigued by the idea of rating by a complexity/generality scale and then correlation of that to the same type of scale given to a search.
As to the correlation I've noticed (in ODP editing) between AdSense sites and "non-usefulness-to-surfer" and to the demurral expressed, the definition of "usefulness" may be part of the difference.
Definitions:
(1) "If this site disappeared from the internet forever, would a single one of the next ten billion surfers miss it?"
(2) "Does this site contribute anything to the sum of human knowledge as represented on the internet?"
(2a) "Does an ODP editor offhand know of, or can easily find, any non-bloody-obvious information presented on the site?"
(3) "Would a surfer find any information on this site that he wouldn't have seen half-a-dozen times looking at a dozen random competitive sites?"
(4) "Could a surfer bound-and-determined to spend money regardless of where, manage to spend that money despite having had to traverse the site on the way?"
It is evident that the overwhelming majority of webmasters are content, yeah EAGER to make bloody sure they fail all but the last test. #2 is, of course, the ODP standard, and #2a is, of course, what the ODP uses in practice.
A couple of examples that I've run across recently: One enlightened me with the critical information that next time I designed a bathroom I should provide storage for towels and soap -- but no mention even of a shelf for the Sears catalog. Another informed me that in order to give a car away I would find it critically important to find someone that wanted it, give it to them, get a receipt, and save the receipt for tax time. There was no indication whatsoever that either webmaster had ever owned a car or used a bathroom.
I can't imagine anybody on earth stupid enough to need advice like this, who could be trained to use a search engine. And those are by no means atypical: if either one of them fell off the edge of the web, hundreds of replacements have already been published and are merely lurking deep in the lower myriads of search engine results, waiting for their moment in the sun.
Give me a website by someone that actually knows something based on their own more-than-typical experience (and most people do have more-than-normal experience in SOMETHING). Raising stinkflowers. Housetraining skunks. Translating medieval German. Building butterfly models. I don't care. I'll read it with amusement and be enlightened despite myself. But ... spare me these blathering ignoramuses.
Go ahead. Check out the search results. Tell me those aren't typical.
Please.
2. Are you an adsense publisher?
Now I am as of late April? Didn't actually get around to putting up the ads until last week.
3. Did you make alot of money from Google traffic?
Yes, but have now brushed the dust off my resume.
4. Dedicated or Shared IP? Do you think switching to a dedicated IP from shared will improve rankings? Dedicated IP. I'd rather not say, not that I really know.
5. Do you believe you were penalized?
NO! Since despite the great loss of traffic, Google's still been nice to me with direct traffic to those money making pages. Problem is, I now realise that most of my customers were those folks simply passing through....I think!, nah, I believe it.
6. Did you forget the 301 redirect, to redirect non-www to www vice versa? (I did!)
Never did the 301 redirect. Tried 2 versions I'd found on WebmasterWorld and they screwed up my SSI parsed pages. So I gave up. HELP!
7. For those of you in the sandbox before Allegra, did Allegra take you out of the sandbox, then did it dump you back in the sandbox in the Bourbon update?
Never affected by Allegra or Florida.
8. How many keywords per webpage do you use when optimizing? How many words per page on average? I don't see things that way. Never have and never will. I write naturally.
9. Proximity of keywords? Are they too close?
I'm always concious of keywords and phrases but it doesn't dominate my approach. (see #8 above)
10. How old is your site?
4.5 years
11. How did you obtain links when you first started out? How often do you get new quality links? Did you stop obtaining links, thinking you would continue to get them naturally?
I continue to get them without asking. Starting out, I was misguided and did a tiny bit of link-farming...But stopped that soon after. 'won't touch it now unless I can get it from a true authority site.
12. Do spammy sites outrank you?
One in particular is on my death list. Beyond that, my regular competition has suffered even worse woes. But then I'm not checking all variations of key phrases applicable to our sites. Only those I've counted on for years.
13. Do you do well in Yahoo, MSN, etc? (yes!)
At MSN I do spectacular. It's almost a carbon copy of the old Google.
As for Yahoo? Errrrr, the pain still stings. 'Would rather not talk about it.
:-((((((
14. Do you hate Google.
I won't hold this against them. Methinks perhaps they're trying too hard these days. Errr, did I mention I'm dusting my resume?
As I was composing my thoughts a jet flying really low passed over the house. Never in all of the time I have lived here have I ever had such an occurance. It set off all of the car alarms in the street and caused me to mash my finger trying to get out of the house<g>. (I thought it was an earthquake.)
Just as I thought I had gotten over it, they came back from the other direction. It was as if the house and then neighbors house were falling down it was so loud.
I got a chance to try out some things that were implicit in your suggestions. I wrote down the top response on google and then one from page 6 and page 9. I am too new to notice the subtle stuff, but even at a gross level, I could see why Google might be returning things the way it does. The one site had 52 pages (more than 500 if you ignored robots.txt) and each page listed all their keywords one time. So there was a nice one to one correspondence between keywords. The graph of their word usage appeared normal (unless you ignored robots.txt) and indicated that there was some normal speech stuff going on.
Page 6 had a screwed up robots.txt file, but if you ignored it, they had a good word spread, but were much smaller.
Page 9 only had two occurances of the keywords and one of them was in the title. They also only had two pages.
None of the sites had links to the outside world. I can see from looking at it that Google did a good job of organizing them.
People talk about how there is a lot of subtleness, but apparently if one looks from those results that hit page one, versus the results that hit page nine, you don't need to consider subtleness.
I don't think I have enough understanding to determine what the differences are between a first place and a second or third place. I do think though that I could identify the site you were talking about, after this exercise, as it becomes apparent that the poor sites don't have good word variety.
Anyway, thanks for the suggestions of a path to follow. This should keep me busy for quite a while<g>.
Well, I tested it out once more on "wholesale computer." Found that again things looked different for the number one site when using robots.txt and when not using it. The site had only five occurances of the word computer when read without the robots.txt and more than twenty five when read with it.
Then I tried a page from search return page 27. It looked like an okay store, it has 155 unique pages, but only 33 of them had the word computer on it. They also had keywords such as shoes, umbrellas, etc.
Besides that, the biggest problem seemed to be their word usage. They had almost sixty percent of their word usage was single usage, and then by the time one got to eighty percent it was up to 30 usages per word, indicating that it wasn't normal human language. It is getting clearer. Again, thanks for the lead.
Site is strongly single focus, good sized, adsense, adwords, dedicated, since late 98 early 99, Forbes Best of Web, Yahoo Coolshades site, Dmoz Editors Pick, depending on which S/E you ask we have somewhat over 16,000 back links, We have been referenced by Google Answers.
This update is still in flight,
I do wish they would do this offline.
I'm sitting here thinking about the new site (should have launched in the first quarter) we are going to push live tomorrow.
We are getting off-topic.
Please move the "insult other peoples web sites, even if you have never seen them" topic to a new thread.
This thread is about Google, not about your ego problems.
Thank you.
Google needs to keep its results fresh. For any reasonable search term there are 1000's of sites that are relevant and each one could justify a top 20 listing.
I search for sailing stuff. If Google served the same sites year on year I would be tempted to search elsewhere. However, now I get variation each month and build up a bookmark list as and when I find sites I like.
My own sites fluctuate, one month high and perhaps next month lower. Nothing changes on the site. The danger is being tempted to change your site seo, which may mean you never catch a top place again when the algo that suits you kicks in again.
Ranking fluctuations are something you cannot beat. You may be hit purely because its someone elses turn at the top.
I am sick of people that keep sucking googles ass. They are ignorant and deserve every bit of bad advertising they get.
1. Are you in a competitive money area?
Not really.
2. Are you an adsense publisher?
Yes.
3. Did you make alot of money from Google traffic?
Up until now I was covering hosting, it just started to go up and I was planning to invest $1500 into site development since I made $750 during last year so I was counting on Adsense to return my expenses by end of the year - NOT!
4. Dedicated or Shared IP? Do you think switching to a dedicated IP from shared will improve rankings?
Shared
5. Do you believe you were penalized? If you put in keywords that only you should show up as, are other sites outranking you?
Yes. I'm been outranked by all the pages that link to me!? Like ones using DMOZ listing! I made some mods for PC games – exclusive to my website – and every other page that only mentions my mod is outranking me!
6. Did you forget the 301 redirect, to redirect non-www to www vice versa?
yep
- Are there any other htaccess codes to put in by the way?
windows server so no htaccess … one of the reasons I planed to switch
8. How many keywords per webpage do you use when optimizing? How many words per page on average?
My keywords are sort of unique so I don't have to optimize much so I thought now my content that is suppose to be exclusive and unlivable anywhere else on the net is being outrank by general news sites with news years old!
9. Proximity of keywords? Are they too close?
Nope.
10. How old is your site?
At current domain since August 2003
11. How did you obtain links when you first started out? How often do you get new quality links? Did you stop obtaining links, thinking you would continue to get them naturally?
Usually link exchange with related websites, added few to Dmoz and other directories plus several to Wiki.
12. Do spammy sites outrank you?
Not so much spammy but ancient pages which haven't been updated in this century got put on top!?
13. Do you do well in Yahoo, MSN, etc?
Relatively, I don't get to many robot visit so they to don't have all my pages, I get high ranking although sometimes it shows to odd page (i.e. forum instead of relevant page) but traffic I get from other SE is laughable it seems most of my visitors use Google.
14. Do you hate Google.
Yes, now even I can't find what I need. >:-)
Google is hand editing and I have several screen shots
to prove it.
Of course they are, and they are changing the algo all the time as well. There is no one factor but a lot more people are probably dropping out due to algo cycles than hand editing.
>Of course they are, and they are changing the algo all the time as well. There is no one factor but a lot more people are probably dropping out due to algo cycles than hand editing.<
Agree 100%
But I can well understand that majority of friends here find it hard to accept.
But friends!
You yourselves are describing in your posts the symptoms of "algo cycles" /"Rotating algos" and you need to take this possibility into account. The sooner the better ;-)
Reading this thread I see I'm not alone - which is a bit of a relief really.
Some questions arising from this thread...
What is this stuff about 301 redirects?
What do you mean by sandbox?
Thanks
People are talking about 'my site' but it was my understanding that google was supposed to be ranking pages not sites. If that is the case do those of you whose sites/pages have disappeared see any change in the distribution of google's referrals to your pages. It would appear to me from the general tone of the postings that whole sites are being hit and since most sites will have a variety of seo factors on their individual pages the conclusion would have to be that 'on page factors' ex. keyword density and 'to page factors' ex a link to a given page must not be the reason for the change.
In order for the whole site to disappear it would seem that
A. Some kind of site wide 'penalty' or 'filter' must be in effect. If so why? This is very hard to believe when sites like danny's and EFV's are getting hit. It would require an algorithm with unbelievably subtle parameters (because their sites are just plain normal good sites like 100,000s of other good sites) which gets it wrong (because their sites are just plain normal good sites like 100,000s of other good sites).
or
B. They are really just the unfortunate victims of a Google screw up. Maybe one of their 'cheap and cheerful' PCs just crashed when those sites were having their PR calculated and they lost the result and decided it just wasn't worth stating from scratch with the calculation to recover the data for a relatively few sites.
I think its something like B. So the good news is you don't need to fix it cause it ain't broke and the bad news is you won't recover till Google does the next update.
Secondly, I've yet to see any evidence that keyword density actually makes any real difference with Google.
I've been tracking the top 30 positions for a keyword phrase for the last month (checking every other day) and also checked the Keyword density of most of them looking for spammers. the top ranking site has over 30% density for that keyword phrase (all 3 words separately are over 20% too.)
The top ranking site is still there (in all 3 major engines too with a PR 6) but the others that had at least one keyword phrase above 30% and the rest about 20% dropped to page 4 so it appears they may have been penalized--for those words at least. They were less PR than the one above. All other sites in the top 30 are not spamming the keywords so it appears that google has cleaned out the spammers except for the one, in this area at least.
An interesting thing re researching this is that sites are bouncing back and forth about 5 positions, i.e., for a site that appears at #6 it will bounce from #6 to #11 every few days and back again. the higher ranking sites bounce less, i.e., 3 to 4, 5 to 6 etc.
Or maybe the pr6 site has perfect hilltop inbound links that overrides any word density algo effecting it.
You cannot isolate any factor, they all work in tandem and have variable effects which are unique to each site.
e.g. Good Hilltop links in = ignore word density
My thinking is that somehow, some large sites got confused by Google as scrapper sites. The reason is that each page in the site had one or more outbound links. This is what exactly an automatically generated scrapper site would have, whether they have scrapped their content from search engines, sites providing free articles, or RSS feeds.
Therefore in this scenerio, one would expect sites to be built on free content, or database driven affiliate sites to be hit particularly hard. If your site has a lot of unique content, then there is no need to have some many outbound links.
A competitor of mine has a duplicate copy of their website loaded on another server. In other words, two exact set of files on two different servers. This same website let their domain expire right in the middle of bourbon and was down for a week. All through Bourbon I have been watching to see if this site would get hit with the much acclaimed dup. content filter, but to no avail. Been holding a top ten position through out...even during the time the domain registration had expired. All my other competitors, including myself have been bouncing around all over the place.
Go figure.
Therefore in this scenerio, one would expect sites to be built on free content, or database driven affiliate sites to be hit particularly hard. If your site has a lot of unique content, then there is no need to have some many outbound links.
I don't think outbound links per se are likely to create any problems with Google. Why? A couple of reasons:
1) Google's core concept, PageRank, is a formula that's based on linking, so it wouldn't make any sense for Google to discourage outbound links.
2) Google's founders and its Ph.D.'s come from university backgrounds where citations are an integral part of academic writing. The Web's hypertext links were conceived as citations (after all, Tim Berners-Lee came from an academic background, too), so it's hard to imagine all those PH.D.'s at the Googleplex looking askance at outbound links.
That doesn't mean that outbound links in combination with other factors or patterns couldn't trigger a downranking, of course.
It seems that cite your evidence or source would be very important to mindset of google. The exception to that rule may come into play when the source being cited is not germaine to the content of the anchor page. But, as always, this is pure speculation as is everything when it comes to the googleplex.
