My sites have not been crawled by GoogleBot in several months.
I was told that this was due to the regular crawling of my pages based upon AdSense placements. This made sense to me, but...
Does anyone else WITH AdSense still see GoogleBot crawls of their sites?
Its very strange but we had almost completely the opposite experience to that which many here are seeing.
Before the 27th our home page could not be found in the site: search and the SERPs were a disaster.
Since the 27th the home page is just where it should be, followed nicely by all the most important pages.
SERPs are getting better and more pages are showing for the site.
We have other sites that didn't have this problem before, and so far are unaffected
Further to my recent post (Last post in old thread):
I have been doing side by side search comparisons with one of the IP addresses I mentioned (old results) against my google toolbar datacentre (new results).
I used a 3 word search term which I knew I ranked well with. On the old IP address (18.104.22.168) I am Number 3.
On the new SERPs I have dropped out (still indexed just not in top 100) BUT most of the top 10 are still there in near identical positions. Given this result (and in my opinion) my page has been filtered by a new algorithm/filter/change. If all the results had been radically different I might believe it was a mistake of some kind.
I'm seeing slightly different ordering on the same 3 word search term when used in conjunction with site.mysite.co.uk when comparing against the old IPs. I'm seeing supplementals on both old and new datacentres.
I'm not a big fan of the current google SERPs but I honestly can't say a google user is going to see much difference in my genre for the same search terms since J27.
I'm now asking myself, If others haven't been affected then why have I.
|Does anyone else WITH AdSense still see GoogleBot crawls of their sites? |
Yes, I still see Googlebot and Media-bot crawling my impacted sites regularly.
Since the 27th, my site has seen an increase of 10-30 pages of non supplemental results per day. It's clear to me, at least for my site, that the index is still undergoing updates every day. I have yet to see any of my newly re-added pages ranking yet though... Big Daddy rolls onward...
I still don't see this as a filter / webmaster penalty as too many people have been effected and from the webmasters I've spoken too most have and some of those are squeaky clean so I'm still waiting for G to put a fix in place plus "if" it is a penalty why would the index be dropped when on other DC's its not been
I hope and pray that you are right.
On this occasion, I would be delighted to be wrong!
|Does anyone else WITH AdSense still see GoogleBot crawls of their sites? |
Yes, every day. Today it seems to have crawled almost every page of my main site. This is in addition to the "mediabot" which crawls for Adsense.
Still got some weird stuff appearing in G's index for 3 out of 4 sites, but the main one appears to be back on track.
|"If" it is a penalty why would the index be dropped when on other DC's its not been |
My site is indexed on both the DCs used for comparison. One appears to be showing new "worse" SERPs, the other old "good" SERPs. I have assumed the index was the same and that it could be a tweaked algo that is the difference.
Still desperately want to be wrong!
I've just spotted something that may be relevant.
2 of the posters here are talking about travel related sites - bontar and donelson mention travel. The one of these that I looked at (by donelson) had a lot of links from their other site. It also had a obvious key phrase name, with a hyphen in it.
Why is that interesting?
Our site that just recovered from this site: hell is travel related, and has a key phrase name with a hyphen in it, and lots of back links from our other travel sites (basically other destinations)
I don't know. But it could be the hyphen thing again - that's been in lots of threads recently - and it could be the spammy site name or the links.
Any opinions on this?
just had it confirmed by b4rney that they are travel related too.
The site: tool shows lots of pages for hotels, which are presumably provided by an affiliate, so this could be a continuation of the drive by Google to hit affiliate hotel booking sites, or affiliate content in general.
We blocked G going to the (affiliate provided) hotel detail pages some months ago - maybe that is what freed us?
last time this happened in March I said it was an attempt for G to remove affiliate based sites and I still think this "could" be the case.
I have started hiding my affiliate links in an effort to hopefully pull my own site back in "if" this is the problem
so many ,if's, maybe's or could'be's its enough to turn a webmaster to drink - anyone want one
>>>enough to turn a webmaster to drink - anyone want one
Tigger - yep that is the plan for tonight.
With ref to G banning/downranking affiliate sites.
Sites will probably struggle if they use affiliate content with no added value as they will get hit by dupe content penalties etc but unless you have lots of links going to an affiliate and little unique content I cant see this would cause any real concerns for Google.
On five of our six sites we have no affiliate links. And on the one that has affiliate links, they are for Amazon and do not appear on every page.
b4rney has told me the hotel detail pages are original content, so it's more dangerous than I thought. Hiding the affiliate thing may be a good idea. We also took the date search tool off our front page, which is along those lines
|I have assumed the index was the same and that it could be a tweaked algo that is the difference. |
I don't consider the index the same when one shows my homepage with the "site:" command and the other does not. Without knowing how their system is engineered I don't have many conclusions to draw...
Since we know the homepage passes things like PR to internal pages and the DCs no longer see the homepage (in my case) then the internal pages no longer have the same value. I don't know. Just speculation.
All of my content is original and unique. My site has not had affiliate links on any of the pages until about 2 weeks ago when I added 3 amazon product links to one of my articles (one page). My logic tells me that Google would probably just drop this article out of the SERPs if it bothered them instead of dropping my whole site. I don't think these changes are about filtering sites that use affiliate links.
|I don't consider the index the same when one shows my homepage with the "site:" command and the other does not. |
Hi tflight. Point taken. I looked at the mcdar dance tool and found some google ip addresses which had differing results to the others when using site:www.mysite.co.uk, that is my homepage was the top result as opposed to some other random page.
I tested these ips and noticed that the SERPs appeared to be similar to pre 27th. I imagine that these few datacentres have not yet rolled over to the new search algo.
I have 15,300 pages listed on one DC and 15,500 on the other. Very similar. I tried a known search term on the DC with old SERPs and found I was still there. Very high. I tried the same search on the new DCs and was nowhere in sight but most other similar sights were.
I have checked and the page in question is included in both DCs index, as is my homepage. I was simply intrigued when I saw differences on the dance tool.
I assume that google can return different results from the same index, which is where the algo's come in. I don't know if your dilemma is the same as mine. I'm indexed, just not appearing in the SERPs since the 27th.
Try it yourself! Let us know if you see pre 27th results on the IP addresses I gave.
|I hope and pray that you are right. |
Unfortunately, B4rney, you are right.
Yesterday, we did a careful side-by-side comparison of the old and the new, and came to exactly the same conclusion: the new results are a carbon copy of the old results, but with some sites' pages surgically removed. The effected pages are usually relegated to page 50, 60, or 70 - always well below scores of completely innapropriate pages including foreign language ones!
This rules out an update and strongly suggests that Google have switched on an over-zealous Spam-filter.
As far as scenarious go, this is about as bad as it gets. If your site is caught in the crossfire, unless your CNN, the BBC, or Amazon, noone is going to notice. Google will be patting themselves on the back right now, because their 50 target spam sites were no doubt delivered a direct hit...Never mind, the many thousands of legitimate sites that have been decimated by a 90% reduction in rank...there's always collatoral damage...right?
|Never mind, the many thousands of legitimate sites that have been decimated by a 90% reduction in rank...there's always collatoral damage...right? |
Yes, there probably is, but it's also worth remembering that having a "legitimate" site is no guarantee of good rankings--or even of being included in the index. For example, an affiliate site with boilerplate product or hotel descriptions might be strictly white-hat, but it could still run afoul of Google's duplicate-content filter--no matter how legitimate it might be from a business and SEO perspective.
Just to get a few things clear -
This is not to do with hyphenated domains, affiliate websites, travel etc - I have seen a complete switch back to pre-Big Daddy, with all the same supplementals, drop in rankings, site: search issues making an exact comeback. I do not belong to any of those areas.
Why this has been effected - now that I do not know. May have been an attempt by Google to fix those areas you mentioned, but it has definitely affected my main site as just a pre-big daddy switch.
And if a google switch to an older index or whatever reslts in all my suplementals coming back, I am sure it will be fixed too. But the permanent fix that we all have been waiting for for is nowhere in site. I have already written off six months of Goog le traffic every year when they go about fixing things and whatnot.
|it's also worth remembering that having a "legitimate" site is no guarantee of good rankings--or even of being included in the index. |
I guess this depends on your definition of "legitimate". By my definition, a legitimate search engine must include, or endeavor to include, all legitimate and relevent pages for a query. A legitimate search engine would then attempt to rank these legitimate and relevent pages in an order that best represents their likely relevence to the query.
All collatoral damage represents a failure to live up to this standard.
Your definition of legitimate seems to include "thin affiliates". These, by Google's own definition, are not legitimate and therefore are not collatoral damage.
|Your definition of legitimate seems to include "thin affiliates". These, by Google's own definition, are not legitimate and therefore are not collatoral damage. |
I don't think anyone at Google has ever said that thin affiliates aren't "legitimate." As I've understood Googleguy's comments, the issue isn't legitimacy, but duplicate content and value for the user.
The same thing would apply to e-commerce sites that use boilerplate product pages. From a search engine's point of view, users aren't being served well if a search for "pink widgets" returns 1,000 identical catalog pages.
It's been three days since the changes.
I am pretty confident this wasn't an accidental data push or anything like that. The results aren't changing on the DCs, so I think this is something that is permanent. Well, as permanent as things have become with Google.
It's just very frustrating not hearing anything official from Google on what happened.
i too was affected, dropped 95% of search hits. on 27th the pages went supplemental in site: command and homepage is nowhere in the list. the page titles are all screwed up too. i have people now finding my content by typing in the name of the site, followed by the keyword. they find me this way, but it's heartbraking.
I have a hobby site listed in 2 separate dmoz categories, listed in wiki and which was never affected by google's shifts, not before 27th. it's not a travel site and it does not have any duplicate content, everything on the site i wrote with my own two hands.
I'm very puzzled about this development. I sent google a contact form, but i have no idea when i'll get something back. site has adsense on it, it is a big earner.
< the page titles are all screwed up too.
This is the one that's puzzling to me. We've had thousands of page titles corrupted by on page text from a 2005 (!) cache of the affected page. In other words Google is displaying a title of:
Real Title + Snippet of on page text from 2005 version of page
All of these pages went supplemental on June 18. We finally seemed to have turned the corner, and some of these pages are starting to come back...
Because of the continued presence of these sorts of obvious bugs, it makes me hopeful that we're not anywhere near done with this update.
It would make sense that much of the change be planned for the July 4 weekend, I think. While America BBQ's, hopefully the Googlers will be patching things up.
Ya know... I wonder what percentage of those sites that experienced the 90%+ drop had AdSense on them.
I really do think that they "shake things up" for sites that have less than 5 or 6 years on the web. If you used AdSense, then then would know almost exactly how much traffic you get from Google (and other sources). If more than say 50% of your traffic came from Google, then they might shake you up a bit.
Anyway, the changes are going to be permanent, at least until the next update, so we need to live with them.
I really hope they're not permanent. But if they are until the next update, it will give me more time to work out any issues and develop a new site i've been working on. Losing 50% of traffic (and revenues) is not easy, but I'll manage.
btw, the site is old, but my domain is only 1.5 years old. the site was hosted since 1999 for free on AOL.
How do we know if these results are permanent?
My site been online since 1998, no adsense, nothing changed, dominant for all keywords and now some are no where to be seen. Got a lot of links coming into a few pages with unique content from high Trustrank sites!
I think Google takes 1 steps back and 2 steps forward while constantly changing and updating. All will be back to normal sooner than one thinks. I also agree that they like to do tests and updates on major holidays.
| This 178 message thread spans 6 pages: 178 (  2 3 4 5 6 ) > > |