Forum Moderators: open
A few moments ago I took a good, thorough look at which pages Google crawls, and guess what I found... I have a search page on my site--the Fluid Dynamics search script FYI--and on that page, nothing else but a search form can be found. Surprise, surprise, this is the ONLY page the great G crawls these days besides one or two index pages for subdirectories. No other pages, even as I am now adding pages with an average of 1 article (can span several pages) a day.
Another disturbing behaviour has been that Google seems to pay absolutely no attention to links coming in to my site. As I am on a Mac, it's far from easy to see the PR on the 400+ pages that link to my site, but with the help of a script, I have been able to determine that I have at least 50 pages with a PR a lot higher than mine. Yet, the great G gives me credit for 13 links only. It has been mentioned on these forums that G only gives you credit for higher PR pages; well, I've found that some 15 pages that link to me have a PR 8, the rest is PR 7 and 6. I haven't bothered looking at the 350 others, as I am not even given credit for the 50 high ones.
Based on these experiences--which have been going on from August and still no change in sight--I have come to the conclusion Google is losing it. A SE serves people who search the web for a certain topic. I'm not going to claim that for the keywords for which I was previously listed as number 1 to 20 (depending on the keyword) my site was the only authoritative site, but I do know that with the current behaviour of the bot, valuable information is not presented to people looking for it.
Instead, when looking at the first 30 positions for the keywords that I target and deliver what is considered by many to be valuable information about as well, I find a disturbing number of sites irrelevant to the search term used.
Luckily for me, I had a Plan B lying in the cupboard collecting dust, so I have stopped worrying about Google and have taken other actions that secure a basic stream of visitors to my site--I'm not even complaining anymore.
I do find, however, that--like software monopolies--search engines monopolies may be a very bad thing. I don't know many webmasters worthy of that title who don't keep an eye on their Google rankings. And, although I know there are those that get much deserved higher rankings nowadays, I can't but feel that Google's quality isn't what it used to be.
Perhaps Bill Gates' weight behind wanting to make MSN Search a real search engine isn't such a bad idea after all...
it hates me...i've come to terms with it now ;)
Cheers,
Jonny
These rolling updates suck for inner content that is updated daily. But there are still deep crawls that happen too.
Google got an algo update back in May by some stanford boys. I'll try and find a link to the article.
If I remember correctly they talk about google and links that count.
Basically those that are still out there going balls out trying to find link partners are just wasting time. A couple of good ones wouldn't hurt but I'd be finding authority sites to put in my daily ariticles. That is yummy spider food :)
Are you saying the majority of your site is uncrawled/unindexed?
Did you not find better search rankings after having had new external inbound links and waiting a month or two? (even if the new links do not show-up as backlinks)
try getting more external links to your inner pages and put internal links to new pages on your pages that get crawled daily.
your "search page" getting crawled most is probably because its one of your site's highest Pagerank pages (nearly every page links to it?)
if I were a search engine bot I would probably also check every link I found on a new page.
smiley added
Yes!
