Broken links and Google

What does G do with broken links on the homepage?

[freejung]

I ask because a former client of mine a few months ago had an amateur make some changes to his site, and the guy deleted a page and left a broken link on the homepage. Subsequently, and I have no idea if the two are relaed, Google stopped indexing the page properly, showing no title or description in the SERPs. Furthermore, G dropped most pages on the site from the index, leaving only about ten or so, not titles or descriptions on any of them.

The strange thing is, the homepage didn't drop much in the SERPs, it's still on the first page for a highly competitive term which took me months of work to get. And there it is, at number five or so, with no description or title, and has been like that for months.

Could this have been caused by the broken link?

I've fixed it now, so we'll see if G indexes it properly again.

[Marcia]

It doesn't sound like that's all that was wrong, just the one broken link. It might be a good idea to run the page through a validator.

I moved some sites a month or two ago, and ended up with one broken link to a page on the site - the page never got put up on the new server. That page only disappeared from the index and lost it's PR, but when I put the missing page back up, it immediately got picked up again. No PR yet, but it will.

[freejung]

Thanks, Marcia, I'll try that. It's odd, though, because it was doing fine before with the same code, unless the guy changed something else that I'm not aware of. I'll check it.

[Chad]

Broken links have no negative effect besides PR-passing opportunity cost of the dead link. Google just assumes that the link is good but the destination page is temporarily unavailable.

In fact. Google goes ahead and assigns a PR value to the ghost page. A neat SEO trick is to create such dangling links to sites or pages that do not yet exist. After a while, when the site or page is actually created, it has a PR value from birth.

[JudgeJeffries]

Chad,
Does that also apply to a site that has robots text barring say Google.
I have such a site and G visits every day but then retreats. If I remove the text will the site have instant PR?

[Chad]

Hi freejung,

I've no idea, but it you try it and report back, we'll all know a little bit more about googlebot. ;)

Just out of curiosity, why do you have a site with a robots.txt disallow?

Regards,
chad

[freejung]

Actually, it was JJ who had the robots.txt, not me. Update: I've validated my page, it had a few missing and improper attributes, but nothing major, certainly nothing that wasn't that way when it was doing fine in Google. I've been told that this is a common issue, and I've seen it on other sites. It just seems strange that it should go on for so long, and I was looking for an explanation.

[freejung]

Further update: I've been playing around with validation, and I have yet to find a single competitor in the SERPs I'm interested in which validates, and many of them have what I would call eggregious errors, such as closing elements which are not open, certainly much worse than anything my page had. I don't think validation was the problem. Sigh... I guess I'll just have to wait and see what happens.

[Chad]

I just got an interesting email from Google stating that they do add pages to their index even when those pages are disallowed by the robots.txt file. They just do not crawl the page.

They say that the only sure-fire way to keep a page out of the index entirely is to use the NOINDEX meta tag.