Welcome to WebmasterWorld Guest from 18.104.22.168
I would like to get feedback from webmasters who experienced huge drop in traffic because of these (caffeine) changes. We lost almost 40.000 daily visitors.
Site is well established and quite old with lot of natutural backlinks, online since 2004.
What do you think ? Are these changes temporary or permanent ?
Most of my sitemaps have thousands of URLs. Right now, I have a small number of sitemaps (14), totaling 58,193 URLs. WMT says only 34,570 of those are indexed. It is next to impossible to deduce which specific URLs are indexed and which aren't, in a huge sitemap.
So, I did a curious test in WMT Sitemaps. In addition to all of my other sitemaps, I set up a small sitemap of just 2 URLs: 1 URL is not indexed, 1 URL is. Curiously, WMT says that both URLs are indexed. Now, I know only 1 is. Maybe they other "is", but isn't showing up yet in the SERPs, but will in a matter of hours/days. But for now, this proves the WMT numbers are not correct. Of course, I would have expected it to say that 0 URLs are indexed, not 2, since it is reprotaing overall so many are not indexed. Weird that it would go the other way in being wrong on a sitemap of 2 URLs. This really makes me wonder about the 34,570 number, is it also way _over_-exaggerated? I don't know...
I have done another thing: my home page gets crawled and indexed every day. This I know because a while ago I stuck in today's date in the title, so I know at a glance from the SERPs when the home page in the SERPs was grabbed from my site - it is always no more than 24 hours old. It usually pops up first thing in the morning. Anyway, I first noticed a specific URL that is important to me missing from the SERPs a few weeks ago. So, I put a link to it from my home page. Within 3 days, that URL was back in the SERPs. So, today I have put a few more links to URLs currently missing from the SERPs on my homepage - not too many, just about 15. I wanto see if using my homepage to push a few specific URLs back into the SERPs will help speed up their recovery.
I will report back what happens to this set of URLs.
Now, one more thing I have noticed is also curious: some of my pages have great ranking right now - all links on those pages are all indexed. I believe I have noticed a correlation between pages that rank poorly right now, and the number of links on those pages that link to pages not indexed in the SERPs. The pages I have checked that rank poorly, all have links to pages that aren't indexed. So, I am trying to force these pages to get back into the SERPs via my temporary homepage links, and then see if that affects the rankings of the initial pages that include those links. The master pages in question used to all rank very well, page 1, #1, #2 or #3, most with double indented listings, and now, all pushed to page 3-8 as single listings.
Oh, and by the way - these pages that are missing from the SERPs - all great pages, great content, etc. etc. All double-checked to make sure they are not dupe content, dupe URLs, etc. they aren't. Just some oddball anomoly that caused them to be dropped from the SERPs completely. If they aren't in the SERPs, google can't list them, and I won't get traffic to them. So clearly, these SERPs being missing from the SERPs is affecting my traffic, and now I am experimenting to see if this is affecting my ranking - a double whammy.
I will report back in a few days once googlebot sorts out my test set, and see if my experiment worked. If it did, well, I have a campaign I need to wage to help push googlebot in the right direction to get all of these URLs back in the SERPs - it will be a drag, but manageable with a projected worthwhile payback at the end.
Feel free to critique and comment on my post. ; )
I've blocked it for years as being a nuisance that slows down page loading. With the new anti-google feeling perhaps more people are blocking it.
But you're probably correct: google is failing as a customer source. :(
Google is busy with something else and indexing sites and new content VERY slow.
Oh MY,, when I do a site:example.com/* I see what the issue is... what is up with the indexing?
I just checked a big site that I monitor and it has 5k pages of unique content not showing up with the /* , I am sure that is causing a bit of pain for them.
As for my blog i have just 1800 pages and for mee too is a little difficult to see what ones are not indexed even though i have managed to track some of them.
It is curious to see how, some of them, have lost rank, while other ones have got it out of nothing.
sometimes i feel like some of them have been penalized by keyword...do not if it is correct but this is a feeling of mine. At this point dont kow what to think. (i have stopped hitting my head in the wall a while ago)
the keywords i used in those posts belonged to the long key and in Adword keyword tool they had around 700 or even less click per month.
I have a last question for you and for everybody in this forum. My articles are republished (just a small excerpt) by officialwindowsmagazine.com....all of them. I have an account right there and i just put my rss. So they republish my rss in the frontpage among other posts...and i get some free traffic...
Is it possible that google is looking at this as duplicate content and, because of this i am being penalized?
By the way Helpnow, my visits seems to be arised a little bit...just a little bit (touching wood now) This is what happened:
I was included in a couple of well-respected paid directories, I got a link from a microsoft page with PR 9 and i was stumbled upon a couple of times, i got dozens of links from different forums (of course i didnt do anything on those posts, it was not me).
Here's a thought: Could GA not be showing you as many results because people are blocking GA?
dstiles: I did think about that one but I've have been able to rule it out.
Unfortunately they have pulled some of my best performing keywords from their top positions to well past the 5th page of the results or even deeper.
(Myself, I found lots of instances of duplicate content, basicaly duplicate URLs which I have always normalized but somehow suddenly found their way back into the SERPs in August. All of this has been fixed in a new way using rel=canonical and noindex tags, and my site is waiting for a new re-crawl from googlebot, meanwhile I have been facilitating the process wherever possible by doing URL Removal Requests. I have seen no across-board movements in rankings, though some individual pages have improved. I have also become aware of a major number of URLs no longer present in the SERPs, but I am unsure of the historical number or if these numbers have been getting better.)
Did you find anything?
I'm still going through all my data and change records.
The last major change I made was adding relevant internal links at the end of each article in this particular section that is suffering. I've done this on other sections of the site but they haven't suffered.
I try my best to build my sites for my visitors and per Google guidelines. I've no duplicate content, all articles are written either by me or my partner. I don't build backlinks as they have all come naturally from visitors, bloggers and even schools.
They only thing that was suspect to me was four amazon links on the sidebar that are nofollowed but I've had these for over 2 years so I cant see that being the issue.
Google Webmaster Tools doesn't show any issues either. I have 150 urls submitted and 147 indexed.
Someone suggested to me that it could be a seasonal change in searches being done. Thankfully I have my seasons mapped out on when to expect big traffic and Christmas is one of them. Compare to last year the traffic on this section really is dire.
...Should those of us who are seeing a decline in visitors from Google expect more of the same and a worsening situation over time?
[edited by: Robert_Charlton at 9:06 pm (utc) on Dec. 14, 2009]
[edit reason] removed link and specifics [/edit]
My sense is that a drop in inner pages that for a period of time acted as a 'support pillar' in the information architecture were removed by Google. This causes landing pages above them to take a ranking hit, since the pages underneath were supporting them internally via links.
For example, a category page of mine which currently is #1, with a double indented listing, has 9 products listed on it, not great content, but 8 of the 9 produts are in the SERPs.
Another category page of mine used to be #2, with a double indented listing, but is now on page 7 - it has great content, 25 products, but 7 of these 25 products ARE NOT in the SERPs right now (I know at least some of them used to be recently).
I have many more categories I've checked - each supports the theory to some degree - the more missing product pages on that category, the worse the ranking.
So, my data right now suppots your thesis that it has to do with internal linking to pages which exist in the SERPs.
I've ben tryign to do an end run on my data, and doign what I can to force soem of these products back into the SERPs to see the affect on the ranking of related pages. It has only been about 36 hours, and so far, the pages remain out of the SERPs, though WMT Sitemaps insists they are in. Crazy. I'm moving pixels here...
The big drop comes from our category pages that lost all ranking within a day, these pages do have a some Backlinks (a few hundreds each according WMT). These category pages ranked for related KW's or related Niches for 5 years or so, all have unique content and are a gateway to the hundreds of unique articles in that category. Some individual articles still rank nicely in some cases, or lost their ranking as well, I don't see much logic in why one ranks and the other not.
Why and how really beats me, many spots are taken over by quite bad sites, or even spammy sites. I start to think the internal link structure has something to do with it, but I can't seem to find the key here...
I am like Bhudda...i am by the river bank wathing the river flowing. I have done everything i could do. in WMT the indexed pages are dropping as well. I am quite surprised that here nobody or at least few people are talking about this.
Then on december, I lost another 10%, and numbers shift daily. One day is at 75% capacity, other day is at 65%... in the past I had steady numbers. Site was also established on 2004.
A new site had over 300 visitors for 4 months. Since december, this blog lost 50% of its visitors. Most sites I have on Analytics report a 15% loss compared to last month.
Over the last couple of week I have been getting a lot of Google Alerts like other members have described on this forum.
The lastest alert I received was for my site and some of the pages that I have seen drop off the serps. Just checking their keywords I can now see them between pages 3 & 5. One page has made it to #4 on the 1st page, before the issue occurred it was on 1st position.
From the results I am looking at I see a lot preferences given to wiki's, forums, article directories and link directories. Before hand the top results where filled with my competitors who provided lots of good relevant content but they too have been pulled under.
Looking at my analytics is depressing and if I cant see an improvement by Wednesday then I give up, I want to enjoy Christmas.
Curiously karkadan, my statistics are accurate in numbers to yours, I don't know which is the problem, since october-november the visits have dropped in about 30%, and checking online tools I see that my competitors also have lost visits sice about the same dates.
A. I had "web history" enabled (personalized search). Therefore, everytime I was looking for a certain keyword on Google Search, my site had a good ranking. When I noticed that a week ago, and searched again, I noticed the sour truth.
B. On IExplorer, I have the google bar, and the search bar that comes with IE. The search bar is configured with Google. Curiously, when searching on both bars, the results vary. Keep in mind that since I'm overseas, the Google bar searches on www.google.com.ar (example), and on the search bar it picks www.google.com (no extension).
I've read here that Google Caffeine was going to focus on specific keywords rather than awarding a domain for a keyword variation. On Google, my site keeps appearing on the number 2 spot for the most important keywords I have.
But after disabling the web history seach option, I noticed that in a keyword variation (my second important set of keywords), I had lost some ground.
None of this makes sense but I noticed I have company on page 5 because there is another website that is also on page in caffeine for the same KW but they're also on page 5 with me in the current SERP.
I just wish some SERP Guru would reply to me and say:
"Hey Sean22, Don't Worry About It, Because Come January Your Site Will Be On Page 1, Just Like It Is In Google Caffeine Now"
Yeah, like there's some kind of consistency to this SERP lunacy! If I made 1 mistake it's that I changed my geographic in WMT. When I first slipped to page 2, I was ranking excellent in the UK,AUS,NZ,Canada but slipped from page 1 to page 2 in the USA. So I changed my geographic area to USA and then I slipped to page 5. So then I changed the Geographic area to no country at all, hoping that I could go back to page 2.