Welcome to WebmasterWorld Guest from 54.146.248.111

Forum Moderators: Webwork & skibum

Message Too Old, No Replies

Is submitting to DMOZ worth the resulting scraper sites?

fuel for scrapers...

     

Prolific

5:24 am on Jul 25, 2005 (gmt 0)

10+ Year Member



I've recently started a new site. Started out by submitting to *only* DMOZ while I finished my site.

I was happily entered into DMOZ - well, happily until recently. My site has now been scraped by well over a dozen scrapers.

My question to WebmasterWorld is it really worth it? I've managed to stop a few scrapers in their tracks as I watch my log files like hawk - but they still are going to get it.

There are other ways to get links, but DMOZ is held highly around here, is there a better way?

Marketing Guy

3:51 pm on Jul 25, 2005 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



I think he means the risk of people using DMOZ as a starting point to find sites to scrape.

Worth the risk? Not having a DMOZ listing doesn't protect you from being scrapped. Having a listing has some benefits which are clear.

MG

wyweb

4:56 pm on Jul 25, 2005 (gmt 0)



Then what does this hsve to do with dmoz?

I suppose I digressed a bit.. flew off on a tangent perhaps? I perk up when I hear DMOZ.

martinibuster

5:12 pm on Jul 25, 2005 (gmt 0)

WebmasterWorld Administrator martinibuster is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



MG is right. The benefits outweigh the headaches. In any case, if you ever start to rank for anything, you'll get scraped coming or going anyway.

Another bother that a DMOZ listing will provide is thousands of link exchange request emails. But that's life.

flicker

6:24 pm on Jul 25, 2005 (gmt 0)

10+ Year Member



Yeah, you're going to get scraped anyway. It's totally inevitable in this day and age. I have an online fiction site which isn't in DMOZ and it still gets scraped. It can be really wonky seeing what nonsensical word combinations each story gets scraped FOR. ;-)

hutcheson

6:52 pm on Jul 25, 2005 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



I also doubt that scrapers are downloading the ODP RDF (although, as has been mentioned, they could.) As soon as a site gets high Google visibility (the scrapers are probably using the Google API, is my guess) then it's susceptible to scraping.

steveb

10:04 pm on Jul 25, 2005 (gmt 0)

WebmasterWorld Senior Member steveb is a WebmasterWorld Top Contributor of All Time 10+ Year Member



I don't think it is accurate to say no one scrapes dmoz. It is common to see one or two scraped dmoz pages tacked on the bottom of a page as bot food in the exact way a Yahoo result page would be scraped. This dmoz content isn't from an rdf dump, and the user has no need of a whole rdf dump. They just want the relevant page.

<prolific is apparently talking about scumbots stealing his site, which is a different thing than what is commonly called "scrapers">

hutcheson

5:17 pm on Jul 26, 2005 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



Yes, dmoz.org is itself just as susceptible to scraping as any other site -- and ODP pages are probably pretty good scrapings, so far as keyword distribution goes.

But ... would this translate into more scrapings of sites listed in the ODP?

I see no reason to suppose it would. Well, other than the fact that there are lots of different species of scraper slime molds, and as none are very intelligent, you could expect a few of them to do almost any particular thing no matter how irrational or unproductive.

But ... that's a risk a website pays for ANY kind of improved visibility -- if surfers can see you, so can scrapers. Who would turn down a #1 position in Google SE results because it might make the site more visible to scrapers? So why apply the same logic to any other kind of visibility? Well, almost any. Directories who shuffle all outbound links through a redirection through their own URL, at least COULD provide you some protection -- which doesn't mean any of them DO, or ever will.

Forumlibrary

10:02 pm on Jul 27, 2005 (gmt 0)

10+ Year Member



I gave up on them. Have not been included for a year now and don't expect to get included either anymore, which does not bother me at all. I used the site in the past but stopped doing so as I find all these categories fairly confusing; too many for me. There is just not enough difference of using DMOZ or a major search engine these days anymore. Perhaps they will come up with some "Baby DMOZ's", the restaurant guide might be a sign for this already.

cbpayne

2:11 am on Jul 28, 2005 (gmt 0)

10+ Year Member



I gave up on them. Have not been included for a year now and don't expect to get included either anymore, which does not bother me at all

Whats that got to do with the topic of this thread?
This 19 message thread spans 2 pages: 19
 

Featured Threads

Hot Threads This Week

Hot Threads This Month