Forum Moderators: open

Message Too Old, No Replies

Good Link Neighborhoods Can Turn Bad

How to notice the change?

         

tedster

1:52 am on Nov 6, 2002 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



I recently had a Google scare which prompts me to write this.

One of my clients has about 1,000 outbound links - professional resources, all on topic when they were set up, and many are reciprocal. We watch for dead links with automation, but no automation would catch this.

One of the domains they link to apparently expired and was quickly grabbed up for a big link farm spamming campaign. We had no idea that this had happened.

Happily, I had just suggested that the client do a hand check of all their links. But imagine their surprise to find their respected professional website was now linking to some rather explicit, age restricted materials.

When the client reported it, I was further upset to see that the tendrils of huge link farm was now in a reciprocal link setup with my client. It looks like we discovered it in time - no PR hit, etc.

But now my concern is that a repeat of this incident might slip by us and result in a penalty.

Is this something that others see happening? Is my concern valid? And more than that, is there any system, other than a regular hand check, that would set off alarms when a domain we link to changes ownership?

tedster

12:34 pm on Nov 6, 2002 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



My understanding is that first the bad neighborrhod (domain farm, etc.) gets identified. I'm guessing that an algo throws out a list of domains for a hand check. If you are linking into one of those domains (and, most likely, if it's also linking back to you) then you may be penalized.

But in the case where a few formerly good links go bad, it's unlikely that your site itself will show all the characteristics of a bad neighborhood. So any penalty, if applied, would not spread to other sites who link to you, unless you all were up to no good!

I think the biggest risk comes when you link to a "bad neighborhood" and a domain in that neighborhood also links back to you. But that's exactly what happened in my case. We did escape harm, however. In fact, our site never built any links with PR in mind -- the only goal was a good resource for site visitors.

Maybe we escaped trouble "because our heart was pure!"

[edited by: tedster at 1:14 pm (utc) on Nov. 6, 2002]

Dante_Maure

12:35 pm on Nov 6, 2002 (gmt 0)

10+ Year Member



<Looks at DaveN's post and then at his own and bows humbly>

Sheesh. I've really got to learn how to say more with less. ;)

jackofalltrades

12:44 pm on Nov 6, 2002 (gmt 0)



>Dante

It was a great explanation for me - i was familiar with the term, but not the story! :)

Back on topic, I honestly dont think there is that much to worry about.

If someone does hijack and expired domain in order to "infect" other sites with link farm stigmata, then Google will just identify it as a new spammy technique and update the algo accordingly.

Obviously there would be some innocent casualties to begin with, but at the end of the day if you operate your site honestly and for the benefits of your user (not the SE's), then Google cant grumble, because that is what they are aiming for.

Unreasonable penalties can be removed, and although spammy sites get away with it for a while, then are eventually caught.

The worst thing a spammer can do to your site is waste your time so you cant update it and add to is as often. Ignore them and theyll go away (or Google will deal with them)! :)

My 2c.

JOAT

Dante_Maure

12:45 pm on Nov 6, 2002 (gmt 0)

10+ Year Member



Currently I only link to 4 other sites

This big site is really vulnerable to ending up with a link going bad.

In this case, you're the vulnerable one not the bigger site.

If one of their links go bad, it's just a needle in the haystack, and they are not likely to be penalized.

If one of your links goes bad it's 25% of your total outbound.

In the conspiracy threat you outline the only one that is likely to be in jeapordy is the satellite site that partners itself with a bunch of link farms... it's not going to pass on that PR0 to the authority.

Unless you are linking to the penalized site personally your site is in no jeapordy.

If you want to rescue yourself from the paralyzing fear you're experiencing, re-read the portion of my post that you quoted... and act on it.

rfgdxm1

12:56 pm on Nov 6, 2002 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



>It's an old story, and I have no doubt that with all of the Mathematics PhD's working at Google, they won't fall for the same ruse.

Heh. I found the text of that story online. Yeah, this PR0 thing does have to possibility of exponentially growing. Obviously Google won't ever let things go that bad. Although, the question is how bad might it get before they stop this? A lot of lesser sites may fall, but it'll be stopped before it get to Yahoo! becoming PR0.

rfgdxm1

1:08 pm on Nov 6, 2002 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



>If you want to rescue yourself from the paralyzing fear you're experiencing, re-read the portion of my post that you quoted... and act on it.

The only site I link to this is really and issue is that big one. The question is whether this is paralyzing fear, or good self-preservation? The only possible saving grace I see is that this one big site is so notable if it went PR0 under the scenario I mentioned, this would be news that would be discussed on Slashdot, Webmasterworld, and other similar forums. Google could do it, but it wouldn't go unnoticed. In fact, if they did then I assure that paralyzing fear would spread like wildfire amongst the webmaster community.

MikeKay

4:56 pm on Nov 6, 2002 (gmt 0)

10+ Year Member



This is a great forum and I've learned soo much within the last 2 months from reading these posts. Before I never gave much thought to "bad neighbors", just linked to similair websites who offered great content. But now the story has changed and made some of us sensitive to the problems of linking to "bad neighbors". Now I check the outgoing links twice a month and remove any links that turned PR0 or grey barred regardless if they provide useful content or not. Don't want to risk the rankings for this website and it's sad having to list websites based on Pagerank now because of the dreaded penalty for linking to bad neighbors. It's time consuming checking these links all the time but I'm sure many of us do it anyway.

Yes, I'm concerned when good neighborhoods turn bad which just happened to me recently. Since last update 6 websites that I linked to got PR0. Seems that they may have been cross-linking and owned by the same person. Cross-linking? 2 months ago I never heard of that word. I removed these links immediately and I hope Google allows some time to remove these links before it decides that I fall into the same category for linking to them. I'm not a SEO and we do appreciate the traffic that Google sends us. Some links start out as being good neighbors may eventually turn bad. I now only consider linking to sites with a PageRank of 3 or higher and won't get surprised if they turn PR0. Yes I stay away from link farms but some of these cross-linked websites are tougher to distinguish. Pagerank went up since last update and I'll have to wait until next update to see if linking to those 6 PR0 sites caused a penalty. Hopefully, Google allows alittle time to remove links to bad neighbors when they just turn bad. If not then It seems unfair.

rogerd

5:08 pm on Nov 6, 2002 (gmt 0)

WebmasterWorld Administrator 10+ Year Member



It's really surprising how fast links can go bad. I have a few sites with largish topical directories, and naturally find 404s cropping up. More troublesome, though, are the Ultimate Search conversions. Google isn't penalizing Ultimate Search yet, as far as I can tell, but I worry that if they do many webmsters will find more than just one or two of their links have become risky. Checking one directory, I found multiple highly relevant topical listings that had lost their original useful content and been converted to a bunch of mostly identical links pages by U.S. Checking 404s is easy, but spotting content changes is a lot more time consuming.

hutcheson

5:25 pm on Nov 6, 2002 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



First, a caveat: Google isn't the perp. The perps are the sleazeballs who use the expired-domain-name-purchase scheme to steal links from established pages. They are the ones who are vandalizing YOUR webpage, stealing its value, by making it a less reliable resource, by associating it with products and services that would hurt the reputation YOU are trying to build -- these effects are visible to your human visitors.

Google is just the messenger.

Secondly, it is not just the small link pages that are victimized. The large directories, even (maybe especially) Yahoo, are being targeted. My personal suspicion is that the $300/annum listing renewal fee was established not to keep Yahoo's revenues up, but to pay for the periodic site reviews required to repair the damage caused to THEIR website and reputation.

And, as at other times in history, the whole economy suffers.

You are right to be concerned. I think probably a "page fingerprint" script is the first line of defense, and that such products will soon be widely available (and cheap!). That will weed out the sub-room-temperature scummeisters, which fortunately will be the large majority. It will not be a complete solution, though: one solution would be for the registrars to impose a year's fallow time on any expired domain. (The problem may not be obvious enough to the the domain-name sellers for this approach to be politically viable.) And, of course, even that is not a _complete_ solution.

Brad

5:36 pm on Nov 6, 2002 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



We also need to distinguish between domains that have switched or gone 404 and sites that have just run afoul Google. A site might be bad in Google, but otherwise perfectly alright in every other sense. I do not think we should let Google hijack the terminology and our thinking.

Just because Google says a site is bad by Google's definition does not mean a site really is "bad". Google may call something a bad neighborhood - but then again Google is looking out for itself in these matters.

All of which is a long way of saying, do not equate high PR with real quality. They are not really the same.

That is the real danger in all this, we start culling on topic and good quality links to please Google instead of to please ourselves and, more importantly our visitors.

rogerd

7:07 pm on Nov 6, 2002 (gmt 0)

WebmasterWorld Administrator 10+ Year Member



I couldn't agree with your last statement more, Brad. Too much link discussion is focused on issues like "should I link to a PR2 site?" or "am I giving PR away if I link to others?"

If you link to good sites that will serve your visitors well, and keep an eye on those links to spot 404s and domain snatchers, I think your risks are low and you will, sooner or later, reap traffic benefits.

hstyri

12:38 pm on Nov 7, 2002 (gmt 0)

10+ Year Member



Back to one issue in the first reply:
We watch for dead links with automation, but no automation would catch this.

Oh, it is. One simple method would be to calculate a checksum based on the content of each page you link to or using other simple techniques to chack the content. If the site has changed, you look up the registry information. If the registry info has changed as well you flag the site for manual review.

If you have a really large index and a budget you could of course subscribe to a daily list of deleted domains. ;)

Chris_R

1:34 pm on Nov 7, 2002 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



There are public domain name monitoring systems on the Internet.

You guys (and gals) worry too much.

Dante is right.

Google isn't stupid. They don't have 3B webpages from banning everyone. They aren't going to penalize you, because you have 3,000 outgoing links and one of them - or even a few - turns bad.

Just imagine taking this to extreme. If two links turned bad on one of the sites YOU link to - they go bad by accident - then you go bad - by the time it is done - everyone on the Internet is banned.

If you want to monitor them - fine - go ahead. If a link gets purchased by someone else - it will probably go dead for a few days anyway due to the way registrars work. You could use a link checking software - which you probably should be using if you link to a lot of sites anyway.

europeforvisitors

2:30 pm on Nov 7, 2002 (gmt 0)



I think Daunte_Maure has it right. Instead of being paranoid and getting into a panic, just link to many quality sites so that--if a few go bad--the undesirable links will represent such a tiny percentage of your link library that Google's spam filter won't be triggered.

In other words, don't be obsessive about defending yourself from Google penalties unless you know you've been leading a life of sin and have a guilty conscience. :-)

web_india

5:17 pm on Nov 7, 2002 (gmt 0)

10+ Year Member



Now the question would be, what that X percent should be.

Only a guess - less than 10%

To keep those % in check, is it a good idea to split your single link page into multiple link pages, if you have more than 20 outgoing links?

I think the biggest risk comes when you link to a "bad neighborhood" and a domain in that neighborhood also links back to you.

I think the penalty, if any at all, will only be levied if those reciprocal links are for the same pages. Not if your link page links to their home page and their link page links to your home page.

web_india

5:20 pm on Nov 7, 2002 (gmt 0)

10+ Year Member



In case you are linking to only a handful of sites, you need to give attention to whom you are linking to.

But if you own a site with hundreds of outgoing links, then you don't need to worry about some of those links going bad.

Also, in worse scenarios, it is more likely that your link page will be PR0, not your whole site.

nell

6:09 pm on Nov 7, 2002 (gmt 0)

10+ Year Member



Getting detected by Google is not all you need to worry about if you're linking to a farm or undesireable site.

One also needs to be wary of any competitor who, given the slightest reason, will report you. You're more likely to get banned as a result of a complaint/s than spider detection.

Dante_Maure

10:41 pm on Nov 7, 2002 (gmt 0)

10+ Year Member



You're more likely to get banned as a result of a complaint/s than spider detection.

As fathom has so often stated... "rubbish" :)

Certainly, you need a site that can stand up to manual review if your're on your way to the top. There will always be folks that cry foul when you leave them in the dust, even with the most benign and legitimate strategies.

That being said, the number of manually applied penalties is a drop in the bucket when compared to the many thousands of sites that get caught whenever Google rolls out a new filter, or tweaks the strength of its existing ones.

Google has repeatedly and emphatically stated that they prefer not to penalize by hand, but instead study their spam reports to better engineer their filters.

Think about it.

If they were to monitor their spam reports and manually penalize each infraction they observe, they'd have no control group to test their spam filters on.

GoogleGuy has said they only hand tweak on rare occasions when the spammer is "really annoying".

Otherwise, you can be sure that they build a file of the sites that are successfully using an undesirable tactic, then reverse engineer a filter to try and squash them.

This process could potentially take a few weeks, if not months to execute without causing a great deal of collateral damage to innocent sites.

This is one of the reasons so many webmasters that are completely self absorbed cry...

"Google's Spam Report Is Worthless!"

"I submitted a spam report about <insert vile spam site> who obviously has no friends or family and hurts small animals for fun because they are using <insert vile spam atrocity here> and Google hasn't done anything... AND... it's been 2 whole weeks already!"

On rare nights when I'm feeling a bit grouchy and have heard this
bleating for the umpteenth time, I wonder if an anti-spam police filter wouldn't be a fun thing to have on some forums. <grin>

So my only real question about GoogleGuy's comments on spam is...

Spam aside... is there a "really annoying" filter in the works, and am I at risk? ;)

nell

4:04 am on Nov 8, 2002 (gmt 0)

10+ Year Member



Google isn't stupid. They don't have 3B webpages from banning everyone. They aren't going to penalize you, because you have 3,000 outgoing links and one of them - or even a few - turns bad.

You're more likely to get banned as a result of a complaint/s than spider detection.

I agreed with Chris_R. and the general consensus of opinion regarding bad links not general spam abuse tactics.

That being said, the number of manually applied penalties is a drop in the bucket when compared to the many thousands of sites that get caught whenever Google rolls out a new filter, or tweaks the strength of its existing ones.

Are you claiming that many thousands of sites get caught for having these few bad links(subject of thread) whenever Google rolls out a new filter?

netcommr

5:36 am on Nov 8, 2002 (gmt 0)

10+ Year Member




I have a script I wrote for automating this process, but don't use it. The script will crawl a domain and extract all outbound links. Runs a head request on all found links. All other than 200 response are flagged for review, then check the 'last_modified' date against database. If changed or no date returned, then check the page and domain for PR level at Google and compare to DB. Any changes, good or bad, or those found to be below threshold are flagged.

(same for inbound links, just have to manually enter them into DB)

If I understand correctly, this is a violation of Google's terms of service so I can't use it. BUT, it is our only way to 'really' overcome this challenge and make sure our links are giving our site the proper linkage.

Google_Guy,
If your company is going to rank our sites by PR through automation, then why can't we check it with automation? I don't want to use a false referrer and UA to do this, but what are my options in the future if we don't have some way to guard against this.

Chris_R

5:55 am on Nov 8, 2002 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



Google_Guy,
If your company is going to rank our sites by PR through automation, then why can't we check it with automation? I don't want to use a false referrer and UA to do this, but what are my options in the future if we don't have some way to guard against this.

Nice try, but I don't think that will work :)
Google owns the equipment and doesn't want people doing automatic queries.
Google won't index your site if you ask them not to.
Google is for users, not webmasters.
Automatic Queries can slow down the system (however - I find this hard to believe it is a big problem, but it could be)

I am not sure what you are even using this script for. What do you mean proper linkage? Do you have agreements with these sites to link to you? If so - you can get a recip link checker.

I doubt you will run into much of a problem if you run the script very slowly, but I don't really see much of a point to these scripts anyway. It is against their TOS.

I don't think google is going to change their mind on this anytime soon. I guess it would be possible for google to start charging for heavy usage of their API that they are allowing limited use of now for free. If they decide to do so - I think they should allow webmasters to do this, but I don't know if they will.

netcommr

6:09 am on Nov 8, 2002 (gmt 0)

10+ Year Member




Chris_R, thanks for the reply, but I already know all that. I was really looking for GGs response and what his thought were on this...

The script started out(year ago) as a PR analysis tool... turned intyo a utility, but never used outside of testing.

2_much

12:43 am on Nov 12, 2002 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



I always see this in terms of "maps" and "percentages". I tend to think the penalites will apply if you're linking to sites that are in a web map.

85% of the sites that I've had penalized are ones that had a closed group of links, without enough links coming in from other sites.

So unless you're running some kind of link farm, I don't think there's much to worry about. But I could be off or naive.

Rockrz

3:18 am on Nov 12, 2002 (gmt 0)



YeeeHaw...Get 'em boys!

NFFC

4:28 am on Nov 12, 2002 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



>in terms of "maps"

Thats the way I see it too. I think it's important to realise that what looks "normal" to an SEO may not be normal in a collection of 3 bil web pages, sometimes you can stick out like a sore thumb.

I look at it this way; it's my site, I choose to buy the domain name, I decide who I host with and therefore who shares my neibourghood, I decide who I want to link to, I add the links to my site, I FTP up and publish my changes to the web.

Whose responsibility is it to ensure that the sites I choose to link to from my site are "good" and continue to remain good?

This 55 message thread spans 2 pages: 55