| 11:13 pm on Apr 19, 2005 (gmt 0)|
Well, a site:www.dmoz.org search has gone from 22 000 scaper pages, last week, to zero a few days ago.
At the same time site:dmoz.org says that there are 11 million results. The real site only has 600 000 categories, and 600 000 Category Charters, and a few thousand informational pages. That makes only 1.2 million real pages. However, yesterday you couldn't get beyond 953 results. Today you can't get past 584 results.
| 11:19 pm on Apr 19, 2005 (gmt 0)|
GG - "we have changed our heuristics for 302 redirects"
Glad to hear of it also.
We have seen plenty of 302 redirects and most have been taken care of. One in particular has hijacked more than 1000 pages at the most we have seen. Since the recent updates the bulk of thes results have went supplemental and at the same time they did ours did.
Our site has been removed from theirs for quite some time actually before these updates (the old 302 redirects are now just redirect to another site). We noticed that a bunch of those old 302 pages have come back with very old cache dates. When this happened many of our pages at the same time have reverted back to and old 301 redirected URL's with caches of our newest design but back from last year(supplemental) and what hasn't reverted have lost title/description (except for recently crawled pages). If any of these old 301 pages would be considered dupe content we are in a world of hurt.(They shouldn't show caches of our new design since that URL has never seen it). It just seems like this thing need to just be crawled out but since all of this we haven't see our normal 4000-5000 requests from googlebot. Just requests for the same few pages almost everyday.
| 12:16 am on Apr 20, 2005 (gmt 0)|
It is all that new math that confuses you, it gets to me as well. I'm finding my graph theory and finite combinitorics, matrix algebra, number theory, advanced calculus and other textbooks are of no help.
Laplace transforms just go poof and numerical methods just give unspecified syntactical errors (funny the subroutines no longer compile)..
Must be canonical math.
I'll have to see if I can find a primer that won't confuse a simple woodland critter. But I'm also rather long o the tooth so I'll probably never figure it out even with a primer.
I hope that all of the babies that got tossed with the dirty bath water get saved, however I wouldn't hold my breath you might get quite blue.
| 12:38 am on Apr 20, 2005 (gmt 0)|
Yes, my site came out of the problem and is doing better than ever. Just hope it doesn't happen again.
I did learn a lot from experiencing it though. I had let my site go stale until this got my attention.
| 12:52 am on Apr 20, 2005 (gmt 0)|
You took action, a lot of folks haven't. Some are still scratching their heads and trying to figure out what happened.
The clock just got started and what GG told crobb305 in msg# 116 of this thread doesn't really bode all that well for sites that haven't taken any action.
I'm glad to hear that your actions have resulted in a positive traffic change for your site.
| 1:27 am on Apr 20, 2005 (gmt 0)|
Kirby/Emmett, I'd love to hear details about the sites you mention. If you could submit the sites in question to google.com/support with canonicalpage in the title and include "Kirby" or "Emmett" so that I can recognize it, I'd like to ask someone to check those two cases out.
E-mail sent. I had only canonicalpage in the title and put my name in the top of the message. Hope it gets through.
Kind of hard to describe a website issue in under 1000 characters though :)
| 1:44 am on Apr 20, 2005 (gmt 0)|
|GG told crobb305 in msg# 116 of this thread doesn't really bode all that well for sites that haven't taken any action. |
Yeah I have been working on this since it (the hijacking) started last May. I have been posting about it since then, deleting unrelated urls with the removal tool, emailing Google, etc. Apparently "declining PageRank" made my site vulnerable to the tracker2s that hijacked my url last fall. When I searched site:mysite those tracker2s were showing as if they were part of my site; my homepage was indexed (searching www.mysite.com) as one of the 302s. Yes, my PR went to zero in Sept but came back to 7 in Dec.
All the 302s are gone, and so still are my rankings. Still waiting to see what happens.
| 2:26 am on Apr 20, 2005 (gmt 0)|
|Anyway, the most interesting question to me is: Are sites starting to return? Has anyone seen their sites come back already? |
I had a 302 redirect problem that knocked my site way, way down in the google serps on Dec 15 and traffic went down even more at the beginning of Feb. I got the site that was using 302 directs to remove them around the beginning of March - in addition to removing some copyrighted content from my site.
My google traffic (which had dropped about 95% by then) started coming back around 3/29. Just a trickle at first. By 4/9, I was seeing a substantial increase. Now my google traffic has come back to about 40% of what it was before this problem. I'm very pleased to see my site coming back :) and I'm hoping for a full recovery.
| 4:08 am on Apr 20, 2005 (gmt 0)|
Thanks for looking into this stuff GG.
| 4:27 am on Apr 20, 2005 (gmt 0)|
Googleguy - in msg#112 you said
|Marval, if someone is doing 302s to your site, you might be able to find redirecters by looking in your server logs for unusual referrers |
Can you tell us how specificaly to identify a 302 hijack?
There are lots of valid unharmful 302's.
We were id'ing them by the ones appearing in site:
Is there a chance that hijacking urls' could still be present but not appearing in site:?
Assuming that site:mysite shows all url's directly associated with mysite - are these 'extra' url's being filtered from the results or are they no longer associated with mysite:?
Could you elaborate on this a little for us?
| 4:37 am on Apr 20, 2005 (gmt 0)|
I used the removal tool to get rid of many 302s to most of my sites, but I can gladly confirm that Google's fix is working, as a couple of the 302s were giving errors and
I could not remove them. Now they are gone.
I had noticed that some of my sites list two urls: [site.com...] and [site.com....] Usually the first without description. I have been using 301s for a long time, but afraid of dup content filter I used the tool to remove the [site.com....] Unfortunately, Google removes the [site.com...] version as well.
I sent a reinclusion request but got a response saying that they don't give personal responses. How can I get those sites back?
| 4:39 am on Apr 20, 2005 (gmt 0)|
Actually, if someone does a 302 redirect to your site most browsers will show you the *original* referrer to the 302 redirecting site, not the 302 redirecting site itself.
searchengine.com has a link to badguy.com, and badguy.com's page immediately redirects to your site (goodguy.com) :)
Goodguy.com's logs will show the referrer as searchengine.com, not badguy.com, if badguy.com does a 302 redirect using http headers rather than a meta "http-equiv" header on a page.
So, to make a long story short, it's nearly impossible to tell who the bad guys are from logs alone.
| 5:08 am on Apr 20, 2005 (gmt 0)|
NOTE: Do not submit your own site to our url removal tool in attempt to force a canonical url. I repeat, do not submit your own site to our url removal tool. Using the url removal tool was some idea that a WebmasterWorld member came up with and started talking about. I just talked with user support about a reinclusion request, and using the url removal tool on your own site will *not* help. All it will do is remove your site for six months.
The user support person asked me to emphasize not to remove your site with the url removal tool; it won't do what you are trying to do.
zeus, I'm not sure if there's a way to undo the url removal you submitted for your own site. I'll ask someone to check it out if it can be done though.
| 5:19 am on Apr 20, 2005 (gmt 0)|
the name is bucaro.
Thanks for the warning, GG. I hope is not too late in my case.
When I search a site with sitename.com and it gives no description, but I search with www.sitename.com and it does. Does it mean is a dup content penalty? or what type of penalty is that?
| 5:30 am on Apr 20, 2005 (gmt 0)|
GoogleGuy, most of my pages were duplicated because of upper-case/lower-case characters. I used the automatic removal tool to remove the upper-case versions. Now, my site has gone from bad to worse in the Google SERPs.
I now know that I did the wrong thing. What should I have done to remove the duplicates?
| 5:37 am on Apr 20, 2005 (gmt 0)|
I would have done a 301 from the wrong-case urls to the right case urls. I'd do a reinclusion request and mention your WebmasterWorld nickname and the circumstances. Then we can ask someone to check it out.
| 5:51 am on Apr 20, 2005 (gmt 0)|
I too did the same thing and now my site along with the bad site which had done a 302 redirect on me dont show up on google. My site only shows up if i search for www.site.com not for sitename or anything else this happened right after i used the google removal tool.
I emailed for reinclusion and got a standard cut paste reply, what do you suggest we do gg?
| 5:55 am on Apr 20, 2005 (gmt 0)|
Good morning from Europe.
I lost 75% of my Google traffic in 3rd feb 2005. Found few 302 redirect which hijacked few of my pages. Got them all removed using Google removal tools.
When run site:www.mysite.dk there were also duplicates which were old redundant files not linked to any page but seems Googlebot found them and got them indexed. Those were also removed thanks to Google removal tools.
Can see now that my site (created 1997) is clean of 302 and dups.
Yesterday I submitted reinclusion request (in case that a spam penalty has been inforced on my site) and received an automated response.
Should I expect to hear again from user support whether a possible spam penalty has been removed?
[edited by: reseller at 6:00 am (utc) on April 20, 2005]
| 6:00 am on Apr 20, 2005 (gmt 0)|
Yup im in the same boat as you, did the exact same thing, my traffic too tanked on 3rd feb.
| 6:15 am on Apr 20, 2005 (gmt 0)|
|NOTE: Do not submit your own site to our url removal tool in attempt to force a canonical url. I repeat, do not submit your own site to our url removal tool. Using the url removal tool was some idea that a WebmasterWorld member came up with and started talking about. |
Unfortunately, I followed that member's advice out of desperation on April 15 after trying many other methods which I think made the situation even worse. I was VERY reluctant at first, but I mean, my site was not even showing up 1st for its unique name or for unique phrases within content pages (since August 2004).
Iím glad you guys are hard at work on the matter. I guess Iíll find out in six months or so for my site. Thanks for reading.
P.S. On google.com/remove.html it says "...90 day removal of your site from the Google index" however once logged in it says "Öremoval system will cause a temporary, six months, removal of your site from the Google index".
| 6:18 am on Apr 20, 2005 (gmt 0)|
RE: Valid vs Bad 302s
GG thanks very much for the very helpful posts.
We've implemented many 301s but still use 302 redirection to count some of our outbound clicks - is this a mistake? Are most people in this thread using any 302s at all anymore? Just static links?
| 6:28 am on Apr 20, 2005 (gmt 0)|
You should block the redirect file (redirect.php) via robots.txt.
| 6:39 am on Apr 20, 2005 (gmt 0)|
"I would have done a 301 from the wrong-case urls to the right case urls. I'd do a reinclusion request and mention your WebmasterWorld nickname and the circumstances. Then we can ask someone to check it out."
Still I am looking for an answer of some sort on the 301 issues I also see. Like many others here we have a 301 redirect fix for www and trailing slash in url's. This has been in place for many years. I see both intermixed in each of the results when doing site:www.widgets.com and site:www.widgets.com. This used to show correctly. Right before we took a dive we noticed that we had 2 index pages one with and one without www showing up with tite description and design being the same even though the one without the www shouldn't be showing anything except url because of the 301. I have tried doing this kind of search on other sites that have this 301 in place and their results are 1 set with www and 1 set without (with www is url only and without www having full title and description the way it supposed to be and how we used to show). It kinda goes with our old deep content 301 redirect urls having new content in serps. I dragged through 5 months of log files and ALWAYS these pages returned a 301 to googlebot.
| 7:00 am on Apr 20, 2005 (gmt 0)|
Wow, I went to bed last night and missed all the action!
On March 9th I sent an email to email@example.com with the subject "canonicalpage hijackers", was it received and reviewed?
I outlined what I consider to be an extremely serious problem regarding 302s and Google manipulation.
Here in Italy I have identified a group of people who have registered hundreds of domains (and probably more like thousands but I haven't got the time to track them all down) that are all "pseudo search engines" which all use the same basic template based on 302 redirects similar to the way Overture works.
These "search engines" have replaced about 2o of my smaller clients in SERPs by stealing their content.
Even exact phrases with quotation remarks (I mean long phrases that belong ONLY to a client's web page) yield results for these templates while excluding the real web site in spite of the fact that the real page IS indexed in Google.
Shall I resend this email detailing more precisely who these people are and how they do so?
What we have here is a deliberate attempt to exploit the 302 bug in Google and distribute the "technique" as quickly as possible in order to profit from all the hard work of webmasters at the expense of honest site owners.
| 7:10 am on Apr 20, 2005 (gmt 0)|
I too removed mysite.com during the 1st week of Feb to recover from duplicate content issue. Removal tool said '90 Days'. Now I hear it is six months?
I've tried the reinclusion request but it hasn't worked.
Sure wish I could get mysite.com back in the serps. The 90 days is soon to expire.
| 7:23 am on Apr 20, 2005 (gmt 0)|
Sailor, it's six months. I know it sucks waiting, but for me at least I know I've tried everything else so there is no regret.
| 7:37 am on Apr 20, 2005 (gmt 0)|
So basically GoogleGuy said,
Our site dropped in Page Rank because of a spam penalty which allowed the hijacker pages to become the canonical pages for our site?
That is why when we remove the 302 redirects with the removal tool it doesn't affect our rankings because the cause was a spam penalty.
In my case, My bandwidth for the month was all used up and my host blocked all visitors to my site and put one of their pages in the place of all my pages which caused a duplicate penalty because every page was the same and was getting a 200 OK error.
So that was the cause of my penalty. Just got to ask for reinclusion now?
| 7:40 am on Apr 20, 2005 (gmt 0)|
<I too removed mysite.com during the 1st week of Feb to recover from duplicate content issue. Removal tool said '90 Days'. Now I hear it is six months?>
<Sailor, it's six months. I know it sucks waiting, but for me at least I know I've tried everything else so there is no regret.>
I wish that GG has posted this message already in february 2005 to avoid such sad situations which honest decent publishers as sailor and msja are brought in:
< NOTE: Do not submit your own site to our url removal tool in attempt to force a canonical url. I repeat, do not submit your own site to our url removal tool. Using the url removal tool was some idea that a WebmasterWorld member came up with and started talking about. I just talked with user support about a reinclusion request, and using the url removal tool on your own site will *not* help. All it will do is remove your site for six months.
The user support person asked me to emphasize not to remove your site with the url removal tool; it won't do what you are trying to do.>
| 7:41 am on Apr 20, 2005 (gmt 0)|
And so have we but at the time given the situation we figured the worse that would happen is the 5 deep linked hijacked URL's we did would just disappear.
If I remember right we also did a few old pages that we removed from our site in February. Kindly rebounded in March, when we did the 5 hijacked URL's, up until the dreaded 23rd update.
Never run it on the main URL and top level directories or even sub directories just 5 deep content pages and a few deep content pages that were removed from our site.
Still I don't believe this could have caused all of this for us especially when one option says...ahem..."Remove a single page using meta tags". Heck who knows.
| 7:50 am on Apr 20, 2005 (gmt 0)|
Good morning.. ciml
GG has posted recently very critical and valuable info which 100s of publishers, who have been subject to 302 redirect issue, might benefit of.
May I suggest to compile all GG feedback in this thread in one document (only read) and post it in a fixed place on top of all threads of forum 30 for a month or two.
Reason is that I don't expect all publishers to read each post of every thread and maybe several visiting publishers haven't realized yet what hit their sites and how to handle it.
Thanks and wish you a great sunny day.
| 8:24 am on Apr 20, 2005 (gmt 0)|
I am really pleased 'the penny has dropped' at google. I felt a touch of warmth towards google this morning when I read googleguys posts.
I do wonder why it took so long though? This is not to be nasty, all credit to them for even bothering to say they had recognised the problem and have made changes.
I will make some guesses, i'd love to know what has actually happened.
- 302's were so deep rooted that they couldn't just make the change due to unpredictable results that might occur. It just a long time to fix.
- "Bigger fish to fry". The problem was not considered that serious.
- The problem mainly effected sites with a 'minor penalty' of some sort so whats the problem anyway.
- Warm fuzzy factor is wearing off so they had to fix it. Negative publicity.
- They didn't know what people were talking about.
- Increased workload, so many people email complaining, they had to look at it even though they didn't think it was particularly interesting. Compounded by more support issues due to people excluding their sites.
- It was spreading with unpleasant results and becoming serious.
Without wishing to speak too soon:- Well done to those that kept the pressure up and sought to help, perhaps you had no choice?