Google search result omits DMOZ listing?

[irock]

Hi, have you guys seen this one?

I keep seeing SERPS with NO dmoz when there was before.

[benc007]

DMOZ has been updating their directory. Google's current cached version of DMOZ data is from 3-4 weeks ago.

[coosblues]

I know the directory I watch has not been updated in over 2 months. Many sites I've added to the ODP show the listing in the SERPs, but Google's Directory is anything but fresh.

[hutcheson]

My guess would be the Google spider is having the same problem accessing the public pages as everyone else is -- so they're simply using the cached pages.

[benc007]

Google's current cached version of DMOZ data is from 3-4 weeks ago.

DMOZ was upgrading their servers when Googlebot spidered their site. As a result Google got old data because DMOZ had their old data online during the upgrade.

[scoobontour]

the dmoz is not crawled like any other site, google (and any other engine that wants to use the data) uses a data export. That happens around once a month, but for a few months while the odp updates were taking place that wasn't happening.

[dmoz.org...]

Paul

[Lorel]

When you click on the submit button on DMOZ's "add url" page nothing happens-- it just hangs there. I checked the code via W3G and it's full of errors. Possibly on purpose?

[sidyadav]

I'd say, its been down for Weeks, I've been trying to add my URL since a month or something but when you submit it, Nothing happens, before you use to receive a "ByPassword" message telling you to enter a Password but now, Nothing happens, Anyone know more about this?

[John_Caius]

If you want to know about problems with specific submissions or the functional status of dmoz.org, this is not the place to ask. The Directories forum charter states:

We are not the ODP help desk. Any specific questions about sites should be taken up through proper ODP channels. We are not affiliated with them, and we have no power to fix them when they go down or change their policies.

The proper ODP channel in this instance is the ODP public forum - you can find it by searching in Google.

In terms of spidering, Google spiders dmoz.org in exactly the same way as it spiders any other site. This is the source of the dmoz backlink if you're listed and do a link:www.yourdomain.com search in Google.

Google also takes an RDF dump from dmoz (rdf.dmoz.org). This is the source of the Google Directory (when Google chooses to eventually update it) and the Google Directory links and descriptions under the SERPs.

Google then also spiders its own directory in exactly the same way as it spiders any other site. This is the source of the Google Directory backlink if you're listed and do a link:www.yourdomain.com search in Google.

The submit function is not working very well at the moment, neither is the public side (what you see if you go to dmoz.org). However, the editor-only server is working very well indeed and previously submitted sites are being reviewed as fast as ever (several thousand new sites listed every day). The server producing the RDF dump is also working perfectly so if Google chose to update their directory tomorrow, it would be a completely up-to-date version of the dmoz database that they would use.

The ODP public forum has plenty of information from the most senior editors on these issues, although there are also some useful comments in amongst the 'me too' stuff in the Directories forum here.

[rfgdxm1]

>the dmoz is not crawled like any other site, google (and any other engine that wants to use the data) uses a data export. That happens around once a month, but for a few months while the odp updates were taking place that wasn't happening.

Incorrect. Check some Google backlinks. I've seen where Google has picked up additions from crawling the ODP directly, yet the site wasn't in the Google directory from the RDF dump yet.

[steveb]

dmoz is crawled like any other site. If they did use the rdf to see backlinks they would be far more accurate about them than they are now.

Backlinks now omit dmoz links simply because Google didn't crawl the page, possibly because Google's crawl is so screwed up in general, but more likely in this case because the dmoz public pages have been inaccessible so often recently. This problem doesn't seem to be fixed so a lot of sites may be missing dmoz backlinks next time too.

[sidyadav]

So let me get this straight.
For a:
Google Directory Crawl = RDF
Google Web Crawl = Simple like any other site

Am I right?

Sid

[DavidT]

I have a site listed in dmoz since February of this year. When searching for the text in the listing on Google it is not there at all. If Google spiders dmoz as per normal they don't do a very good job of it.

[steveb]

There is no such thing as a "directory crawl". They use the RDF to make the directory. The serps are based on a crawl of the public pages.