Welcome to WebmasterWorld Guest from 54.227.14.23

Forum Moderators: Webwork & skibum

Message Too Old, No Replies

Is submitting to DMOZ worth the resulting scraper sites?

fuel for scrapers...

     
5:24 am on Jul 25, 2005 (gmt 0)

Junior Member

10+ Year Member

joined:Mar 20, 2005
posts:86
votes: 0


I've recently started a new site. Started out by submitting to *only* DMOZ while I finished my site.

I was happily entered into DMOZ - well, happily until recently. My site has now been scraped by well over a dozen scrapers.

My question to WebmasterWorld is it really worth it? I've managed to stop a few scrapers in their tracks as I watch my log files like hawk - but they still are going to get it.

There are other ways to get links, but DMOZ is held highly around here, is there a better way?

3:51 pm on July 25, 2005 (gmt 0)

Senior Member from GB 

WebmasterWorld Senior Member 10+ Year Member

joined:Jan 16, 2003
posts:1922
votes: 0


I think he means the risk of people using DMOZ as a starting point to find sites to scrape.

Worth the risk? Not having a DMOZ listing doesn't protect you from being scrapped. Having a listing has some benefits which are clear.

MG

4:56 pm on July 25, 2005 (gmt 0)

Senior Member

joined:Aug 12, 2004
posts:1781
votes: 0


Then what does this hsve to do with dmoz?

I suppose I digressed a bit.. flew off on a tangent perhaps? I perk up when I hear DMOZ.

5:12 pm on July 25, 2005 (gmt 0)

Moderator from US 

WebmasterWorld Administrator martinibuster is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Apr 13, 2002
posts:14208
votes: 219


MG is right. The benefits outweigh the headaches. In any case, if you ever start to rank for anything, you'll get scraped coming or going anyway.

Another bother that a DMOZ listing will provide is thousands of link exchange request emails. But that's life.

6:24 pm on July 25, 2005 (gmt 0)

Preferred Member

10+ Year Member

joined:Aug 28, 2003
posts:560
votes: 0


Yeah, you're going to get scraped anyway. It's totally inevitable in this day and age. I have an online fiction site which isn't in DMOZ and it still gets scraped. It can be really wonky seeing what nonsensical word combinations each story gets scraped FOR. ;-)
6:52 pm on July 25, 2005 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:Jan 30, 2001
posts:1739
votes: 0


I also doubt that scrapers are downloading the ODP RDF (although, as has been mentioned, they could.) As soon as a site gets high Google visibility (the scrapers are probably using the Google API, is my guess) then it's susceptible to scraping.
10:04 pm on July 25, 2005 (gmt 0)

Senior Member

WebmasterWorld Senior Member steveb is a WebmasterWorld Top Contributor of All Time 10+ Year Member

joined:June 20, 2002
posts:4652
votes: 0


I don't think it is accurate to say no one scrapes dmoz. It is common to see one or two scraped dmoz pages tacked on the bottom of a page as bot food in the exact way a Yahoo result page would be scraped. This dmoz content isn't from an rdf dump, and the user has no need of a whole rdf dump. They just want the relevant page.

<prolific is apparently talking about scumbots stealing his site, which is a different thing than what is commonly called "scrapers">

5:17 pm on July 26, 2005 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:Jan 30, 2001
posts:1739
votes: 0


Yes, dmoz.org is itself just as susceptible to scraping as any other site -- and ODP pages are probably pretty good scrapings, so far as keyword distribution goes.

But ... would this translate into more scrapings of sites listed in the ODP?

I see no reason to suppose it would. Well, other than the fact that there are lots of different species of scraper slime molds, and as none are very intelligent, you could expect a few of them to do almost any particular thing no matter how irrational or unproductive.

But ... that's a risk a website pays for ANY kind of improved visibility -- if surfers can see you, so can scrapers. Who would turn down a #1 position in Google SE results because it might make the site more visible to scrapers? So why apply the same logic to any other kind of visibility? Well, almost any. Directories who shuffle all outbound links through a redirection through their own URL, at least COULD provide you some protection -- which doesn't mean any of them DO, or ever will.

10:02 pm on July 27, 2005 (gmt 0)

New User

10+ Year Member

joined:July 27, 2005
posts:4
votes: 0


I gave up on them. Have not been included for a year now and don't expect to get included either anymore, which does not bother me at all. I used the site in the past but stopped doing so as I find all these categories fairly confusing; too many for me. There is just not enough difference of using DMOZ or a major search engine these days anymore. Perhaps they will come up with some "Baby DMOZ's", the restaurant guide might be a sign for this already.
2:11 am on July 28, 2005 (gmt 0)

Preferred Member

10+ Year Member

joined:Aug 14, 2003
posts:573
votes: 0


I gave up on them. Have not been included for a year now and don't expect to get included either anymore, which does not bother me at all

Whats that got to do with the topic of this thread?
This 19 message thread spans 2 pages: 19