Forum Moderators: open

Message Too Old, No Replies

The REASON Dmoz has NOT been able to Update its directory since SEPT

Open Directory RDF Dump not working - New sites cannot be SHOWN !

         

aaaaa

5:23 pm on Dec 9, 2002 (gmt 0)



:(

Search database last updated on: Sat Sep 28 00:54:00 PDT 2002



A Dmoz listings will often help your Google keyword ranking, thereby increasing your rankings on:
Yahoo
Aol
Netscape
Earthlink etc...
The above message was at the bottom of every search result on Dmoz for WEEKS... until Dec 4th.
:(

Search database last updated on: Tue Dec 3 16:22:30 PST 2002

This new message appeared for a few days after Dec 3rd
:(

Search database last updated on: Sat Sep 28 00:54:00 PDT 2002

This message is now again present.
Based on explanations from corresponding with Dmoz editors'...

The Dmoz "RDF dump" has not been working, "THOUSANDS" of websites HAVE actually been Added or Modified since the last Update.

The RDF dump will allow members of the public - with Licenses to "freely" download the ODP index if they agree to their terms.

It takes Dmoz one week to verify if the updates are successful - If not- the last udated date will again re-appear at the bottom of the search listings

There is a Dmoz url where one can check anytime to see if an individual directory has been updated.
[dmoz.org...]

[edited by: aaaaa at 6:15 pm (utc) on Dec. 9, 2002]

dvduval

8:36 pm on Jan 2, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



So why not make a copy of the database and work from that to make the RDF Dump?

DB 1 = the db used to create the new RDF (0 operations per hour)
DB 2 = the editable DB (1000's of operations per hour)

coconutz

8:49 pm on Jan 2, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



>>I'm wondering how much traffic you guys usually see from Google's directory once it's updated

I'm listed very high in the Google directory, but see very little traffic from this listing, and I would guess that this is the norm.

g1smd

9:14 pm on Jan 2, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



I think that the only reason that most [business] users want an ODP listing for, is simply to get an increased PageRank. Most of them probably don't give a hoot about the directory, it's content and structure. Just list me applies in many cases.

aaaaa

10:37 pm on Jan 2, 2003 (gmt 0)



The WebSites that have been listed on Dmoz has shown a significant increase in "PageRank" and "Keyword Ranking"
on Google - ONCE GOOGLE GETS AROUND TO ADDING IT IN IT'S GOOGLE/DMOZ DIRECTORY.
Sometimes - just due to a Dmoz/Google Dir. listing -the WebSites jumped 2 pages for popular keyphrases.
Dmoz has many directories that use all or part of their RDF Dump and occassionally referrals have steadily come from them - according to my referral logs.
I get about 1 to 4% of all my referrals from Dmoz,Google Dir. and the affilliates that use dmoz.
However, some of my sites get over 1000 search engine referrals daily - so the extra traffic makes a difference especially in ecommerce sites.
Possibly people who chose Google Dir or Dmoz represent a more "detailed" or "learned" searcher - so it becomes a matter of quality as well as quantity of visitors.
Google uses it's web search title as opposed to the dmoz search title in it's directory - but uses the Dmoz descriptions, so you may get better keyword results if someone "searches" the "Google Directory"

Dumpy

10:50 pm on Jan 2, 2003 (gmt 0)

10+ Year Member



"So why not make a copy of the database and work from that to make the RDF Dump?"
DVDDUVAL

If you listen closely you could have heard the people at Netscape slap their heads and utter "DUH!"!

When I heard they were going to hand correct the database about a month ago, when they had 11 errors, I knew there was no hope.

I am moving on to crawling and creating my own directories...without attribution. I do not wish to be associated with such an irresponsible organization.

arc_light

11:18 pm on Jan 2, 2003 (gmt 0)

10+ Year Member



Possibly people who chose Google Dir or Dmoz represent a more "detailed" or "learned" searcher - so it becomes a matter of quality as well as quantity of visitors.

Good point - hadn't thought of that. I'm not expecting nearly as many from the directory as from the improved PageRank...but maybe the quality will be better. (Well, assuming they get the RDF working!)

motsa

3:14 am on Jan 3, 2003 (gmt 0)

10+ Year Member



I do not wish to be associated with such an irresponsible organization.

But you have no problem taking the work that this irresponsible organization has assembled and using it?

kctipton

3:58 am on Jan 3, 2003 (gmt 0)

10+ Year Member



Starting to sound to me like someone's Business Plan does not have a Worst-Case Scenario backup plan.

rfgdxm1

5:09 am on Jan 3, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



>But you have no problem taking the work that this irresponsible organization has assembled and using it?

ROFL.

dvduval

5:54 am on Jan 3, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



I really want to see DMOZ get everything straight. There are many volunteers who have devoted a large percent of their free time making it arguably the best human edited index on the internet...

BUT...

I know it's not a search engine, but think about those search engines who failed to update for long periods of time:
Altavista, Direct Hit, Excite, etc.
Where are they now?

C'mon DMOZ! I'm pulling for you. Bring in the big guns! Get the help you need.

europeforvisitors

8:14 am on Jan 3, 2003 (gmt 0)



aaaaa wrote:

The WebSites that have been listed on Dmoz has shown a significant increase in "PageRank" and "Keyword Ranking"
on Google - ONCE GOOGLE GETS AROUND TO ADDING IT IN IT'S GOOGLE/DMOZ DIRECTORY.

Google is still crawling DMOZ even if it can't get the RDF dump to update its directory, isn't it?

It would be nice if Google's directory were up to date, but if you're concerned about having your sites found by the Googlebots and inheriting PageRank from ODP listings, the crawl should be all you need to worry about. (It does take time for crawler data to make its way into the Google index, but that has nothing to do with the RDF dump--that's just the Google index's normal lead time.)

amznVibe

9:22 am on Jan 3, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



a little DMOZ trivia, apparently this is how well
aol/netscape thinks of its free offerings, one domain
alias for DMOZ is [dirt.netscape.com...] LOL! -aV-

aaaaa

4:13 pm on Jan 3, 2003 (gmt 0)



[dmoz.org...]


The ODP staff is dedicated to constantly improving the directory, and your feedback
is important to us. Please let us know about any new features & services you'd like to
see added. We'll work hard to incorporate your feedback.

-- the Open Directory team

First name:
Last name:
Email address:

Subject:
Comments:
_________________________________________________________

Dmoz has a FeedBack Page, I have sent them a link to this Topic - it seems to have save me HOURS of typing

KEEP THE FEEDBACKS COMING! ;)

g1smd

11:11 pm on Jan 3, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



>> "So why not make a copy of the database and work from that to make the RDF Dump?" <<

The copy will have the same errors as the original; doesn't help does it? The RDF-producing script is still gonna fail.

>> When I heard they were going to hand correct the database about a month ago, when they had 11 errors, I knew there was no hope <<

You don't actually know exactly what those errors were and how they were caused. Believe it or not, I hand correct entries in the database all day long -- that is what editors do. The errors needing special correcting by staff all seem to involve URLs with special characters in them (if you look at [dmoz.org...] ); some of the directory uses ISO-8859-1, some of it UTF-8, and other bits under /World use many other encoding schemes. Maybe the original many thousands of errors were corrected using some sort of script, and rather than running a script again it was quicker to just hand-edit it.

>> Google is still crawling DMOZ even if it can't get the RDF dump to update its directory, isn't it? <<

Yes it does. Sites added at [dmoz.org...] soon appear in SERPs at [google.com...] but will not appear at [directory.google.com...] until they grab a new RDF dump from the ODP (they used to do this about monthly, so they have missed ~3 at this point).

>> aol/netscape thinks of its free offerings, one domain alias for DMOZ is [dirt.netscape.com...] <<

Hmm, can't remember what the t stands for, but the dir is for directory.<snip>

[edited by: NFFC at 2:01 am (utc) on Jan. 4, 2003]
[edit reason] As per TOS [/edit]

kfander

1:58 am on Jan 4, 2003 (gmt 0)

10+ Year Member



I guess I'm atypical. When searching for information, I'll browse the first page, but if I don't find what I'm looking for, I'll usually jump to page 5 or so. And I much prefer clicking through directories than reading through the nonsense that I so often get from search results.

kctipton

5:25 am on Jan 4, 2003 (gmt 0)

10+ Year Member



I agree with that.

aaaaa

7:52 pm on Jan 4, 2003 (gmt 0)



for several months now there has been a discrepancy of one PageRank point for many of the WebSites that are listed in
Google's Directory and the PageRank given for these same site that appear on the Google Toolbar.

Sometimes the PageRank of the "Dmoz" Subdirectory is so high itself that the link cause a major major PageRank increase for the lucky WebSite that gets listed.

The trick is to get a close to the Major directory as possible every subdirectory down represents a PageRank decrease. (Dmoz) has a 10 PageRank.

I am wondering if a website that is listed ALSO gets credited for BOTH the Google (Directory) AND the DMOZ links.
which is why it can dramatically affect the ranking.

Ironically, google links to the Google Directory for Websites that are listed in Dmoz, but Yahoo links to the Yahoo Directory that it is listed on. Aol has the appropriate Dmoz links at the bottom of their Search Results.

amznVibe

2:57 am on Jan 5, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



I can definitely see, now that the Google full scan is over, they actually scanned new DMOZ but didn't import it into their own database (I know, it needs to be imported fresh)

The two sites I have with new DMOZ entries actually show backlinks from DMOZ (I added DMOZ on the their links page at the last second before the scan) but of course they don't show the extra line for the Google directory yet.

Patience patience I guess. -aV-

aaaaa

5:03 pm on Jan 6, 2003 (gmt 0)



As a method of calculation, for the sites that NOW have the backwards links Only to DMOZ but NOT yet to the GOOGLE/DIR - Jot down your Current PageRank - again, when the sites get listed on AOL jot down your PageRank.

Finally, Once the sites are on GOOGLE/DIR take notice of your NEW PageRank (both on the toolbar and the one on Google's Directory) if you notice any significant increases - this may indicate how much an effect each listing has on your ranking.

Finally, make a note of the PageRank of Both the Dmoz & Google SubDirectory "itself" that and a particular WebSite is on.

I personally have found a 2 page jump for popular keywords,
once Google ADDS that Dmoz "description" at the bottom of the website's regular keywords. And, have noticed for Some Sites there are more visitors collectively from Dmoz and Google DIR, than from AltaVista Lycos, Dogpile etc.

aaaaa

4:26 pm on Jan 17, 2003 (gmt 0)


Does submitting a site to Dmoz AGAIN BERFORE the website has been reviewed - HURT?

Submitting a website AGAIN to Dmoz after some time has passed may be disadvantageous.

According to some editors it Puts the WebSite in Question to the Back of the que again.

Dmoz does have a helpful site that enables you to query editors about the STATUS of a submitted website.

They usually respond within a few hours - even if you tried emailing that catagory's editor and got no response. http://www.resource-zone.com/postlist.php?Cat=&Board=status

hutcheson

4:48 pm on Jan 17, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



>>Does submitting a site to Dmoz AGAIN BERFORE the website has been reviewed - HURT?

>Submitting a website AGAIN to Dmoz after some time has passed may be disadvantageous.

>According to some editors it Puts the WebSite in Question to the Back of the que again.

Technically, submitting a site again TO THE SAME CATEGORY simply deletes the original submission and replaces it with the new one.

This may help, if the new submittal has better description (from the editor's point of view) than the old one.

This may hurt, if the editor strictly reviews sites in order of submittal date (some always do; others never do) OR if you ask about your site submitted "6 months ago" -- if they see it was just [re]submitted yesterday, they may say "give us a break, hey? wait a month before asking!"

Don't worry too much about it.

aaaaa

8:15 pm on Jan 24, 2003 (gmt 0)




Search database last updated on: Mon Jan 20 04:13:57 PST 2003

I have been getting the above footer under Dmoz search results for several days - however, the "actual" SEARCH results don't seem to bear this out!

g1smd

10:54 pm on Jan 25, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



Yep, it is a glitch caused by the last RDF failure on that date. Search "thinks" there is a new file, but isn't using it as the RDF was not pushed.

zooros

5:49 pm on Jan 27, 2003 (gmt 0)

10+ Year Member



the rdf dump failed again -
that means we have to wait another month ...

"RDFs not pushed
RDF tags used in the current RDF files (2003-01-19)"

g1smd

10:16 pm on Jan 27, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



>> that means we have to wait another month ... <<

Another month? You're kidding aren't you? Where did you hear that mis-information?

The RDF takes about 5 or 6 days to complete each time, and often seems to be restarted about 2 to 4 days after the last one completed (whether it worked or not).

Sometimes the RDF exits without completing, but usually you will see some sort of output within 10 to 12 days or so whatever the outcome. There have been occasions when it was much longer, but this was because the ODP had major hardware and software changes that interrupted the usual timetable for many days.

aaaaa

6:07 pm on Jan 29, 2003 (gmt 0)



while waiting for an increasingly hard-to-get DMOZ listing and UPDATE - there are many other directories with high pageranks which can collectively help ones website significantly:

www.joeant.com

www.backwash.com

www.goguides.com

wwww.zeal.org

www.hotrate.com

www.mavicanet.com

Many of these have pageranks of 6 or 7 and their links can help a website "collectively" as much as DMOZ.

aaaaa

6:08 pm on Feb 3, 2003 (gmt 0)



Wed Jan 29 20:50:26 PST 2003

MANY NEWLY ADDED WEBSITES ARE NOW - FINALLY SHOWING UP IN DMOZ SEARCH RESULTS

[dmoz.org...]

RDF tags used in the current RDF files (2003-01-19)

However, these updates have not been reflected - YET - in Google's directory.

g1smd

9:30 pm on Feb 3, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



They found a way to use the "failed" RDF (it has a lot of duplicate CatIDs in it) to update the search. The "no-CatID" RDF can be found at [rdf.dmoz.org...] now. Google used to download the RDF file towards the end of each month. I have no idea whether they can use the "no-CatID" version or whether they will have to wait for the next "full" version.

g1smd

9:33 pm on Feb 3, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



They found a way to use the "failed" RDF (it has a lot of duplicate CatIDs in it) to update the search. The "no-CatID" RDF can be found at [rdf.dmoz.org...] now. Google used to download the RDF file towards the end of each month. I have no idea whether they can use the "no-CatID" version or whether they will have to wait for the next "full" version.

aaaaa

4:47 pm on Feb 5, 2003 (gmt 0)



If it is'nt one thing with Dmoz - it is another.

The Dmoz search has not been working dependably for days

The Open Directory search is currently under a heavy load. Please try back later.

Now their editor's forum is giving database error messages.

SQL ERROR: Unable to do_query: SELECT t1.B_Number,t1.B_Parent,t1.B_Username,t1.B_Posted,t1.B_Last_Post,t1.B_Subject,t1.B_Main,t1.B_Status,t1.B_Approved,t1.B_Icon,t1.B_Reged,t1.B_Counter,
t1.B_Sticky,t1.B_Replies,t1.B_Rating,t1.B_Rates,t1.B_RealRating,t2.U_Color,t2.U_Status FROM w3t_Posts AS t1, w3t_Users AS t2 WHERE t1.B_Topic = 1 AND t1.B_Board = 'status' AND t1.B_PosterId = t2.U_Number AND t1.B_Approved = 'yes' ORDER BY t1.B_Last_Post DESC LIMIT 26
Can't open file: 'w3t_posts.MYD'. (errno: 145)SQL ERROR: Unable to do_query: SELECT COUNT(*) FROM w3t_Posts AS t1 WHERE t1.B_Topic = 1 AND t1.B_Board = 'status' AND t1.B_Approved = 'yes'
Can't open file: 'w3t_posts.MYD'. (errno: 145)

Imagine this happening to Google or Yahoo with as much frequency - the owners should take this search directory more seriously than they apparently are.

[edited by: Laisha at 10:29 pm (utc) on Feb. 5, 2003]
[edit reason] remove side scroll [/edit]

This 73 message thread spans 3 pages: 73