Forum Moderators: Robert Charlton & goodroi
Is anyone, experiencing a significant drop in pages indexed ?
Or is this discussion purely about serps and not quantity of index pages.
[edited by: tedster at 8:06 pm (utc) on Nov. 26, 2008]
In October, I experienced record traffic. On November 2, I was visiting my live stats program and watched as a plug was pulled.
whitenight - I don't mean to create a rant, but would be interested in hearing your opinion on redirects since I think this change may have something to do with my fallout. My only current plan is to work like crazy on new inbound links to /directory/ because I think Google isn't trusting my sitewide change.
whitenight - I don't mean to create a rant, but would be interested in hearing your opinion on redirects since I think this change may have something to do with my fallout. My only current plan is to work like crazy on new inbound links to /directory/ because I think Google isn't trusting my sitewide change.
You "get it" judy.
<slightly longer rant>
Google doesn't "get" redirects. Not now. Not ever. Anyone who remembers the 302 adsense gaffe of whenever knows this. Anyone who has done even the SLIGHTEST blackhat knows this and uses it to their advantage.
They put bandaids and other salves on the problem but its CHRONIC. It's not going away anytime soon and every new update is a chance for them to expose the unhealed disease.
If those 301s weren't in place from the get-go there's a good chance Goog is going to f#$#% it up sometime down the line.
Better to leave pages as they are and funnel traffic and PR as this has multitudes of SEO benefits as well.
<slightly longer rant over>
I could see teh 301 as a concern if they were new, buy these redirects have been in place for almost 4 years.
Gees I have heard the merits of 301's preached at SES and SMX for years, but lately I'm hearing the opposite.
If you understand how Goog makes algo changes then lets suppose they decide to use "historical data" to turn up the "weight" of links that are 2 or 3 years old --
but turn down the link weight of links that are 4+ years old as "too stale",
and links that are 1 year old as "not mature enough".
Add into the mix, that SINCE Goog knows they don't really know how to solve their redirect issues, they OVER-COMPENSATE to weed out the millions of blackhat 301s and 302s who continually come up with new ways to exploit the problem.
What you have is a very very simple explanation for what's going on and a recipe for a big mess that can wipe out one's rankings when employing redirects.
-----------
As I mentioned before, I would suspect they will add another "bandaid" after the New Year, but then again, Maybe not.
Some of the advice above is more suited to an entrepreneurial site or an exclusively online business (so-called pure-play). But a flagship site for a big brand that also operates offline has some other requirements that come into play. For example, if a change in the platform is needed then some redirects to handle URL changes may be unavoidable, /Judicious use of SOME 301 redirects to preserve important backlink power can be essential to avoid lost traffic.
Certain other kinds of redirects are also very helpful - such as the no-www to with-www redirect. But there are some key points to appreciate:
I've been involved in some corporate redesigns where we redirected only 40 out of 1,000+ urls. Results were the traffic and rankings barely flickered. Google just indexed the new site and dropped the urls from the old site that became 404. I strongly doubt that redirecting every url would have had the same fast result.
I've recently seen this approach called an "SEO myth" by some Google employees. However my experience still tells me that the approach works faster than lots of 301s would.
[edited by: tedster at 5:44 am (utc) on Nov. 26, 2008]
For example, if a change in the platform is needed then some redirects to handle URL changes may be unavoidable
I still think they would be better served leaving those pages as semi-orphaned and using those pages for SEO landing pages.
But Ive had this argument before to little avail.
A couple years ago I had one client who allowed us to do that when they went from html to asp - and it went pretty well. We included a link on the new home page that said "Archives" (it pointed to a copy of the old home page) and we just let all the old urls stay live. Plus we modified the old menus so the old urls had some links that pointed back to new urls - things like Home, Contact and Newsletter Signup.
We also added an "archived page-not maintained" graphic to all the old content pages, but there was not a single redirect in the entire redevelopment project!
Google had no problems with it. At first, it brought in even more traffic because of the new .asp pages going live. Then over about 6 months, many of the original urls went supplemental, but even today, many of those old urls still bring in search traffic.
One point to add is that the way in which Google handles 301 redirection for large, inbound link-rich websites is much different than how it handles these for smaller websites that are new, untrusted, or ultimately simply have lower page rank.
This, of course comes into play often, especially when comparing domains and the effectiveness of implementing a redirection plan.
I am sceptical about whether this affected the tap being turned back on and think it was rather coincidence.
I still use 301's on old pages as well as http://example.com to http://www.example.com and even www.example.com/index.php to www.example.com
I see quite a few sites just disable non www access altogether which results in a 'address not available' and no headers being served at all. I dont see any other alternatives left apart from duplicating the site on www. and non www.
One piece of advice I would give is get a real 404 page ! If you've got a 301 catch-all to a dynamic page served as a permanent page (a soft 404) then you have affectively infinite pages on the domain. I believe google wants to see hard 404s!
How many of you guys still experiencing problems have soft 404s ?
[edited by: tedster at 7:18 pm (utc) on Nov. 26, 2008]
[edit reason] switch to example.com - it can never be owned [/edit]
A couple years ago I had one client who allowed us to do that when they went from html to asp
Have you used clean URIs on the new build ? I mean like
site.com/about-us and not site.com/about-us.asp
That way the next rebuild (to php :):) ) there will be no need to rewrite any URIs !
[edited by: matWright at 7:36 am (utc) on Nov. 26, 2008]
If this were an intentional slap (and not a glitch) that these smaller sites (that redirect to folders of the main site) would also disappear from the SERPs.
Apart from that, if you are clean, have checked your backlinks, outbounds (using Xenu), server logs, used tools to pretend to be googlebot, and otherwise verified your sainthood, plus filed reinclusion requests, then Google Groups is probably the way to go.
Generally, if your site is banned, either you should know why (cos you're very naughty), or you have been hacked, or have serious technical problems. If none of these things apply, then its really a bit of a fishing expedition in giving advice.
@Whitenight/Tedster- while 301s may be problematic, surely they would not get a site banned?
I see quite a few sites just disable non www access altogether which results in a 'address not available' and no headers being served at all. I dont see any other alternatives left apart from duplicating the site on www. and non www.
Well all my positioning came back last night and has stayed. Every client's keyword that dropped to page 10+ is back in it's previous positioning.
Veto, Were you a Oct 31-Nov 4 issue or a this week issue?
@Whitenight/Tedster- while 301s may be problematic, surely they would not get a site banned?
Because spammers are always trying to make their redirects LOOK as "natural" as possible, redirects that ARE natural can get caught in the
"Kill all the cockroaches and sacrifice some healthy tomato plants"
mindset that Goog has adopted with this issue.
Search engines will do no such thing. They will show what the server gave them. In the case of a chain it is indeterminate (in advance, on a sample of one) what the result will be.
This is a poor choice of words by the hosts. They are obviously looking to shift the blame elsewhere for their shoddy infrastructure giving poor results.
Are you absolutely sure about that? Yahoo shows the URL in the SERPs without the trailing slash, but the href link URL usually does include it.
You need to click one of those links and examine with Live HTTP Headers exactly what happens, and also find that entry in your raw server logs, and confirm exactly what happened, in terms of additional redirects, after the click.
While the dramatic drop from the SERPs coinsided with the October SERP changes, there was a programming error on the site. Code that was written to avoid duplicate content issues that had gone wrong and placed a "nofollow" tag on the home page (d'oh!).
No code was checked during this period, but none the less, the problem was all on us.
placed a "nofollow" tag on the home page
Ouch! Thanks for letting us know. It's a good example of how something basic can be the cause of a problem rather than something esoteric or completely in Google's court.
I sometimes read Google Groups to pick up hints from the Google staff there. Many times the issue under discussion turns out to be a site error of some kind. So when I'm troubleshooting a ranking drop, I always work through a few basics first - server headers, robots.txt, .htaccess, meta tags, severe mark-up accidents and the like.
------------------------
Now back to the topic of this thread... ahem...
believe we were discussing the long range impact of the update.
Somebody mentioned AI? Wha?
lol =p
I don't code the site, but I should have looked at the D---pages.
Have you used clean URIs on the new build ? I mean like
site.com/about-us and not site.com/about-us.asp
I wish. I pushed for it, and they did take a look at ISAPI Rewrite. But bureaucracy being what it is, they wouldn't install any rewriting utility for IIS. Now they're looking at a switch to .aspx and STILL they don't get it.
Somebody mentioned AI? Wha?
It seems clear to me that Google is working toward something that they call AI - some kind of machine learning that can inform and change the algo directly. It's a vision they've had for a while - I guess they never watched the Terminator movies ;)
So as I tried to explain above, my idea is that their data structure is modified to allow experimental AI efforts to run easily, in parallel to the production data. I'm NOT saying that AI is really with us, just that the back end at Google may be supporting that kind of experimention. So sometimes troubling changes we notice in the infrastructure may have that purpose in mind.
It's a vision they've had for a while - I guess they never watched the Terminator movies ;)
I'm NOT saying that AI is really with us, just that the back end at Google may be supporting that kind of experimention. So sometimes troubling changes we notice in the infrastructure may have that purpose in mind.
Unless Goog can get some enlightened people on staff, they're gonna have a tough time getting over this particular hurdle.
The 301 problems come from improper use, done wrong and or to many redirect combinations to reach the correct page.
301's are a very useful tool and should be used if necessary and not feared. I did over a 1000 pages with very positive results on a site with 1600 pages.
301's are a very useful tool and should be used if necessary and not feared. I did over a 1000 pages with very positive results on a site with 1600 pages.
Dont misunderstand me. 301s done correctly are fine 99% of the time.
I do think that they are better ways of using "old or changed" pages for SEO purposes but thats my personal preference.
301s done incorrectly and Goog's problems with 301s are a problem (ala sophisticated blackhat techniques).
As said before, I didn't originally want to FUD the board with my 301 rant.
Just trying to diagnose motw general issues and get him thinking on solutions.
Do you see a datacenter that is more current? Or looks like the future?
