freejung - 4:41 pm on Jul 19, 2010 (gmt 0)

MrFewkes, forgive me, but I think you may be massively overthinking this.

If you were to put the same amount of effort it would take to write such a crawler as you describe into finding a better way to get links, I bet you'd have orders of magnitude better results.

If you've already established significant relevance for a term (because your page is about that term, right), the kinds of tweaking you're talking about are going to have some very minor impact on your relevance score, when it's entirely possible that you will be unable to move up in the SERPS with any amount of improvement to your relevance, without improvement in your offsite factors.

I would take the lack of movement as an indication that you're barking up the wrong tree with on-page optimization, and focus more on other factors.