Forum Moderators: open
This update should lead to significant changes in the serps in a month or so asthe crawler now finds new categories, sees some sites deleted and now actually finds those sites that were added since March.
Additionally, in general the Google Directory now carries a significantly higer PR than DMOZ (exceptions will occur where the linking is different). Most obviously, the main page of the Google Directory is PR10 while the main page of dmoz is PR9.
Sites that were added or removed in since March should now feel a major impact (good or bad) in PR -- which of course is dwarfed by the influence of anchor text in the serps, but helps a LOT in terms of being crawled.
I just tried the category at:
http://directory-in.google.com/Top/Regional/Europe/**************/*******/******/*************/
and the page returns just:
ok
I do a "view source" and it says just:
ok
with no HTML tags at all. Just:
ok
What is going on?
In fact every category at [i]directory-in.google.com[/i] does the same thing now.
[1][[b]edited by[/b]: g1smd at 11:48 pm (utc) on Nov. 2, 2003][/1]
Quite possibly. However, how could a page that Google didn't spider because of robots.txt have any PR? If Google didn't spider dmoz.org because of the robots.txt, they wouldn't know of any editor profiles, because to find them requires they spider it.
Google regularly crawls the dmoz.org pages, but regularly has been missing thousands of pages for months. A lot of that has to do with the dmoz public pages often being inaccessible for quite sometime. Since a lot of bots were banned from crawling recently, the dmoz public pages are close to infinitely more accessible, so they should show more consistently as backlinks next update. (The backlink info shown now seems about a month old, and the dmoz public pages were more hit and miss at that time.) Google still is not doing a great job crawling big sites of middling or low-ish PR, so it may still miss large sections of dmoz and the the Google Directory too next time, but in both cases there is reason to hope that the crawl and backlink results will show significant accuracy/improvement next time.
This is very possible. Googlebot would crawl dmoz.org the same as a surfer. If dmoz.org was slow, it is quite possible Googlebot couldn't crawl the whole thing. This would explain Google missing some ODP pages.
If Google didn't spider dmoz.org because of the robots.txt, they wouldn't know of any editor profiles, because to find them requires they spider it.
They are caching the profiles and show PR etc. I believe they have made a change (tweak directed at editor profiles) - profiles are not showing up as backlinks as they were before and are not passing PR. This started last month with many that I noticed. I would expect sites listed in editor profiles to drop in PR very soon (as many already have).
This I would believe. I suspect that Google has for a long time ignored links in ODP editor profiles. Just like guestbooks some time ago, just because Google showed them doesn't mean they counted them. However, if it was a robots.txt issue banning Googlebot, then all the editor profiles should have no PR. If you entered the editor name of a meta into Google, if Googlebot hadn't spidered the ODP, Google wouldn't know the URL of their editor profile. This doesn't seem to be the case.
However, how could a page that Google didn't spider because of robots.txt have any PR? If Google didn't spider dmoz.org because of the robots.txt, they wouldn't know of any editor profiles, because to find them requires they spider it.Google has all the information it needs to give a PR for a page without ever visiting it. That's why in the serps your profile page shows up at number 1 (for your dmoz username) but with only the url for the listing.
They have sometimes not been showing up as backlinks, or showing up as newhoo backlinks, for the same reason that many dmoz pages themselves didn't show as backlinks -- Google has been crawling dmoz very poorly for several months.
LOL, reading most of this topic it seem like many are not going to let facts get in the way of a good story :o)
Dave
Hm, since when do we use such words?
> My ODP editor profile just hit PR7.
Congrats to your profile success but the fact remains - google has not crawled the odp since months. At least google didn't crawl three major categories i know of and didn't follow, index or pass any new pr to new sites listed at these categories since at least july. No doubt no matter what words you use to argue against that fact. It's now the first time that new sites are indexed due to their new dmoz back link.
While it might be popular in many quarters of WW, the "downgrading" of certain sections of the ODP (including backlinks from some editor profiles) probably has had more to do with the technical problems than with any nefarious Google scheme to do away with the value from either DMOZ or the Google Directory.
“The simplest solution that satisfies all of the facts is usually correct.”
A good rumor never hurt anybody but editors profiles are obviously still passing PR and being crawled.
And you are saying this because?
I have no doubt they are being crawled/cached - only they are no longer passing PR (watch closely in the next few weeks).
Do you see PR5,6,7,and PR8 editor profiles in the backlinks for sites? If you do let me know because I checked quite a few and they are not. I am quite familiar with several editors sites and have watched them drop drastically in PR - recently. They are no longer showing their editor profiles as backlinks.
Of course. They are all over the place, most obviously in the backlinks of the categories they edit. And if you look at profiles with personal homepages (rather than full-blown website/domains) linked from them, it is easy to see that the profile PR is the root of whatever PR the personal page has.
Google still crawls them, Google still PR's them, Google still passes the PR, Google still shows their backlinks. Google's crawling is sometimes poor due to its own problems and those of dmoz being inaccessible last month and before, but those are just normal errors. I suppose the errors are quite extensive so it could look like a change, but there are tons that are showing.
In any case, doesn't make sense to draw any conclusions about this until next month, after Google has had a full month to crawl the dmoz pages when they are working properly.