homepage Welcome to WebmasterWorld Guest from 54.167.185.110
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member

Visit PubCon.com
Home / Forums Index / Search Engines / Directories
Forum Library, Charter, Moderators: Webwork & skibum

Directories Forum

    
I have a Dmoz link to me.
I'm puzzled as to what this really means
Event_King




msg:478560
 1:58 am on Dec 15, 2005 (gmt 0)

Hi people,

After checking my aw stats, it seems that dmoz has spidered me or something.

9 different refering search engines Pages Percent Hits Percent

DMOZ 1 0.3 % 1 0.3 %

-----------------
Does this mean I'm now being reviewed by dmoz or have I been indexed (now in dmoz directory).

Couldn't find me site in dmoz by searching, so I dunno why dmoz is showing in me logs......

 

hutcheson




msg:478561
 7:27 pm on Dec 15, 2005 (gmt 0)

I would have thought you meant "raw" stats but the line you quoted was obviously cooked -- we don't know what your stat cooker treats as "DMOZ references."

If we knew what the actual referer lines for the individual references were, our speculation could be more informed, but still not necessarily conclusive.

Event_King




msg:478562
 7:41 pm on Dec 15, 2005 (gmt 0)

AW shows 'Links from an Internet Search Engine - Full list'

Then simply lists each different referring search engine.

I thought this meant where I had links? AW stats shows:

Pages Percent Hits Percent
330 6.7 % 330 2.4 %

Links from an Internet Search Engine - Full list
- Google 225 225
- Yahoo 55 55
- AOL 22 22
- MSN 18 18
- Other search engines 6 6
- T-Online 1 1
- Overture 1 1
- Netscape 1 1
- DMOZ 1 1
330 6.7 % 330 2.4 %

I presume the numbers (for Google, Yahoo etc) correspond to the amount of links pointing to my site from the engines?

Some of these stats programs are not user-friendly and tough to interpret.

texasville




msg:478563
 8:17 pm on Dec 15, 2005 (gmt 0)

Event-King
I don't believe that what you see is spidering. It's actually referring links. I had mine show up with a dmoz referral and about 3 days later I was in dmoz.
However, if you are searching using their search tool on dmoz, it probably won't show up for a week or more. You would have to go to the appropriate category and search thru the listings.
Of coursr...you might not have made it in...soo...but keep good thought.

Span




msg:478564
 8:18 pm on Dec 15, 2005 (gmt 0)

AW shows 'Links from an Internet Search Engine - Full list'

Isn't "Full list" in the above line a link? Shows you all referring URLs in a new window.

Event_King




msg:478565
 9:09 pm on Dec 15, 2005 (gmt 0)

Yep, 'Full List' links to all the SE's that have viewed me site. Search engines crawl my site every day and fetch about 3000 pages every day.

I don't think the 'Links from an Internet Search Engine' is the same as 'Links from an external page (other web sites except search engines)'. It seems they are different anyway.

All these stats are just for December only. Maybe dmoz viewed my site or something, but I'm more interested in why a dmoz link showed on my stats - it's very strange......

Event_King




msg:478566
 9:13 pm on Dec 15, 2005 (gmt 0)

Oops forgot to ask - what are referring links? Is that like another site telling another site to list mine, kind of like a recommendation?

choster




msg:478567
 9:26 pm on Dec 15, 2005 (gmt 0)

If you follow a link on site A to site B, site A is the "referrer" and will show up on site B's "referrer log." However, the referrer is not always reliable; it can be stripped or changed by proxy servers for instance.

hutcheson




msg:478568
 9:48 pm on Dec 15, 2005 (gmt 0)

"Referer" (which may be the same thing as "referring links") is a standardized (but optional) part of an HTTP request header. It's a URL which is conventionally the page (if any) containing the link.

Users and firewalls may block referers, which just means those requests don't include that line -- in other words, you don't know why the surfer chose your URL to visit. And HTTP requestors (bots or browsers) may spoof referers, so that you don't know whether that page REALLY contains a link to you. (That's the "referer porn spam" trick also alluded to above -- 'bots request your page with their doorway page as referer, hoping that URL will get into your logs and from there to Google.) But most of the time for normal browsers, the referer is accurate.

Your log analysis program is undoubtedly classifying page requests looking at the referers and using some sort of heuristic rule.

Event_King




msg:478569
 10:42 pm on Dec 15, 2005 (gmt 0)

So some doorway site is giving out false information about me then? Maybe trying to get me banned? Now I'm panicking.

Is there any way to stop this abuse.......

bether2




msg:478570
 2:33 am on Dec 16, 2005 (gmt 0)

So some doorway site is giving out false information about me then? Maybe trying to get me banned? Now I'm panicking.

No need to panic. It seems very possible that the referrals did come from ODP. Your site could be listed but not be searchable yet with the ODP search engine. It's worth checking again in a few days. Or, if you submitted your site, then you could browse to the category where you submitted and find more accurate info that way.

[edited by: bether2 at 2:39 am (utc) on Dec. 16, 2005]

Key_Master




msg:478571
 2:37 am on Dec 16, 2005 (gmt 0)

Could be Robozilla. Dead link checker.

[robotstxt.org...]

hutcheson




msg:478572
 2:39 am on Dec 16, 2005 (gmt 0)

Event-king -- the whole point is, we don't KNOW that any of the above AREN'T happening -- but we don't have any reason to believe they ARE happening either. It could simply be a visit from an ODP editor.

But if so, is it just an editor surfing to the site, or reviewing the site for a listing, or using it as a bad example in a discussion, or investigating abuse related to some other site, or stealing your links, or stealing your design for their own site, or plotting world domination with you as the first of many disposable victims? Who knows? In other words, in that respect those hits are no more or less omenous as any other hits on the site.

Don't get paranoid. I know of no reason to suspect conspiratorial activity. But even knowing the exact referer string, I wouldn't try to guess the actual intent.

Event_King




msg:478573
 1:40 am on Dec 17, 2005 (gmt 0)

I'll try not to get paranoid.

g1smd




msg:478574
 10:34 pm on Dec 17, 2005 (gmt 0)

IF your site has been added to the listings, then it would show up in the public version of the category within about a week, and later on would show up in ODP search after another few weeks.

Event_King




msg:478575
 11:50 pm on Dec 17, 2005 (gmt 0)

hmmmmm, I've seen this 'dmoz' referrer twice now - in my logs and I haven't been added to the category I chose, nor does my site appear in the results.

I say someone from dmoz is winding me up and visiting my site with no intention of reviewing it or indexing it. Why do people do these things, they must be very bored to play such tricks.

hutcheson




msg:478576
 12:30 am on Dec 18, 2005 (gmt 0)

EK, relax already! There are maybe a dozen or two editors who read these threads, and none of us even KNEW you ever looked at your logs until you just told us.

Oh, someone may have done a "partial add to reviewed" -- which does create one of those odd-looking referers -- just to look at OUR logs to see if any OTHER editor had done anything with the site. That happens sometimes when someone is talking about the ODP status of a site. (And if there really is one of those very rare problems, we can just silently fix it.

hutcheson




msg:478577
 1:06 am on Dec 18, 2005 (gmt 0)

Let me try once more. I'm really not trying to arouse paranoia -- just trying to recognize ignorance.

Any SINGLE event in your logs could be what it looks like, or it could be something different. Most of the time, events are simply what they seem: so if you're looking at large numbers and trends, the logs do what you want.

But trying to get too much information about one link (which, after all, may NOT be as it seems) isn't useful.

ODP referers aren't any different than others. They may be genuine -- they usually are... but any one may be someone doing something odd. If one IS someone doing something odd, it's more likely to be a rogue spider doing something sneaky, than an ODP editor picking one site out of tens of millions -- just on the off chance that the webmaster may be puzzled by the log entry (and beyond that, concerned about puzzlement.) Because most people recognize fairly quickly, "oh, well, another apparently insignificant event I don't understand in the world", and let it go at that. And that's the thing to do here.

If you ARE in the mood for puzzling over ODP entries in the logs, chew on this fact: obviously the ACCESS event should come BEFORE any editing event (the editor looks at the site first, then decides what to do about it.) So WHATEVER you can get out of the log entry, it WON'T tell you what the editor hadn't yet decided to do. It CAN'T even tell you whether the editor decided to do anything -- things get interrupted and decisions get deferred. And, (not so obviously) it can't even tell you what sort of decision the editor was planning to make (which is not necessarily the same as the one he ended up making or not making) when he first visited the site.

And finally, an editor in heavy editing mode may open multiple browser tabs to review a site in one tab and edit its listing in another. That kind of review won't show in the logs at all -- there wouldn't necessarily BE a referer. And some editors may be behind firewalls that routinely block referer. So just because there is NO event in the log, doesn't mean a site wasn't reviewed.

In other words: there isn't any reliable information here. It's just a record of a computer event which may have been part of some (unknown and unknowable) human action possibly related to the ODP.

So: no real information, therefore nothing to worry about. What you don't know can't hurt you here.

bether2




msg:478578
 3:11 am on Dec 21, 2005 (gmt 0)

I say someone from dmoz is winding me up and visiting my site with no intention of reviewing it or indexing it. Why do people do these things, they must be very bored to play such tricks.

I've seen other people fall into this way of thinking when they were anxiously waiting for their site to be listed.

However, from my somewhat limited experience, I would say that it's highly unlikely that someone at ODP is purposely trying to "play tricks" or get you wound up.

Event_King




msg:478579
 4:32 am on Dec 21, 2005 (gmt 0)

Although it would probably do my site some good to be in dmoz, the 'presence' of this referal thing has foxed me. It's very strange and after a 3 year wait to get into dmoz - I'm naturally wondering what the heck is going on. Very frustrating indeed, as I like to be in the know, and dislike the fully unexplained.

With the number of bots and home-made robots scurrying about, it makes me nervous and think sometimes that anyone could go about winding others up. After all, they do it by infecting PC's with nasty viruses etc.

hutcheson




msg:478580
 5:48 am on Dec 21, 2005 (gmt 0)

At most, it means some editor (probably) visited the site. But if so, neither the identity of the editor nor the purpose of the visit is recorded anywhere.

Event_King




msg:478581
 1:17 pm on Dec 21, 2005 (gmt 0)

Thanks.

flicker




msg:478582
 9:59 pm on Dec 21, 2005 (gmt 0)

Sometimes when I'm too tired to do high-brainpower editing, I'll go through a category and click on everything that's been submitted there, and then delete the spam and broken URLs, move misplaced sites, move things that aren't in English to the appropriate non-English categories, etc., and leave the sites that look good for an editor who's more familiar with the topic to review and describe. It could have just been an editor doing something like that.

Event_King




msg:478583
 4:46 am on Dec 22, 2005 (gmt 0)

Just searched dmoz and am convinced I'm not listed. Searched various related categories for my site, all to no avail. This is indeed a mystery. Personally, I reckon it's some kind of bot doing the rounds in an attempt to mess with good folks heads. Either that or I've simply not been listed yet.

I do have a strong theory why someone would wind others up, entertainment value - but until I see evidence of my site in dmoz, I'll think the worst.

Just out of curiosity, can a site listed in dmoz, be searched by url or company name?

lmocr




msg:478584
 5:57 am on Dec 22, 2005 (gmt 0)

You can search by URL - but you have to drop the www.

For example "www . example . com" would be searched as "example . com"

You can also search by Company name.

However, the search feature can lag behind listings by a few weeks sometimes.

badass101




msg:478585
 3:06 pm on Dec 22, 2005 (gmt 0)

E_K - do bear in mind that an editor could have popped in a few times (causing the referrers) and then decided not to list your site in DMOZ, or move it's catgeory as Hutcheson suggested.
So, you may well not see anything on DMOZ, possibly never if the editors don't think your site is appropriate - but you may still see referrals as the editors/servers do their things.

Relax! It's Christmas!

hutcheson




msg:478586
 4:19 pm on Dec 22, 2005 (gmt 0)

I'd rethink that "play with minds" theory. Remember, when the editor visited your site, how could anybody know you'd look at one log entry that closely? Most people don't. It seems you have software that pulled out that one line -- but most people don't.

And also remember, we don't know what the editor might have been intending. I OFTEN visit a whole group of sites with no intent to list any of them -- that is, I'm going through unreviewed suggestions, trying to eliminate the obvious spam and redirect the obvious missubmittals.

So the only question I ask is "is this site OBVIOUSLY spam so I can delete it now?" and "is this site OBVIOUSLY in the wrong place?" I spend as few seconds as possible looking for an obvious answer to the question. If I don't find an obvious answer I leave the suggestion for later review for a listing and for exact placement.

It's just triage. And it's easier (more efficient) to work on just one kind of activity (just deletes, just reviews of promising sites) on just one kind of sites. Remember, editors' reward is a sense of accomplishment: and nobody focuses on efficiency like someone trying to accomplish something.

Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Search Engines / Directories
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved