repeated 'page not found' results

[sssweb]

I have a message board installed on my site (like the one here at Webmaster World) and repeatedly find 404 ("Page not Found") error codes in my server stats. The URL's always follow the same pattern:

www.mysite.com/board/board/page.php

The correct URL should be: www.mysite.com/board/page.php

I've checked my pages over & over for links to "/board/board/..." but never find any. My internal link codes (from pages w/in the message board) are like so: <a href="/board/page.php">link</a>

The site works fine when I navigate, and I've never received any email complaints, so I'm wondering if this is from some bot that spits out bad URLs for whatever reason.

Any ideas? (PS - My site redirects these errors to the correct page, but I'd still like to eliminate the repeated error if possible.)

[rocknbil]

It's got to be a relative link somewhere?

[g1smd]

See if WMT lists any links pointing to those duff URLs.

Run Xenu LinkSleuth over the site to see if it finds anything.

Who requests the duff URLs? Look in your logs.

[sssweb]

rocknbil: are you asking me or telling me? If you're asking, I'd say yes, since by now I'm sure I have no absolute URLs like that.

g1smd: I can't find anywhere in WMT that gives that info; can you be more specific?

I've checked the links w/other diagnostic tools -- no problems.

My host doesn't post those details in my server stats, so I have no way to know. But it seems from the sheer number of them that it must be from an automated source, either a search bot or spammer.

[sssweb]

I think I know what it is -- since my relative links start with "/board/..." some bots(?) are trying to add that on to the directory URL of the current page. So if they're on:

mysite.com/board/page1.php

and read a relative link, they strip off the /page1.php (leaving "mysite.com/board"), then add my relative link beginning with "/board/..."

I thought browsers/bots are supposed to go back to the root directory for relative links beginning with "/" but maybe some don't.

I'll take out the "/board/" in my link code and see what happens.

[g1smd]

You should link by using a URL that begins with a slash and counts from the root.

The place to look for these errors in WMT is in the "crawl errors" section. You're relying on Google having seen those URLs, and having recorded the URL of the page that supposedly pointed at that duff URL.

You also need the raw log files for your site, not some noddy "stats" data. The logs are usually in a folder called logs, above the web root, when you look via FTP.

[lucy24]

I'll take out the "/board/" in my link code and see what happens.

Don't do that. The links are correctly formed, so it's the other end's fault if they're not getting it right. Google does a lot of demented things, but it knows how to read links. It's only when they're getting the link from some third party that they fail.

If your host does not allow you to see your raw logs, change hosts.

[sssweb]

Thanks, I'll check into all that...and leave my code as is.

[sssweb]

I checked WMT - only 1 crawl error listed, and it wasn't the /board/board/ issue.


I also checked my logs. Almost all the bogus links are from foreign sources (crawlers?). Examples:

RIPE Database query service (3: Ukraine, UK, Poland)
Internet Service Provider in Hong Kong
China Unicom Jiangsu province network
LG DACOM Corporation (Korea)

And the referrer is invariably a page on my site with the same URL error; it's also not just "/board/board/" -- sometimes it's "/board//board/", sometimes it's "/board/board/board/".

Obviously it's not on my end; should I just ignore it?

[lucy24]

How badly do you want to keep unwanted robots happy? :)

(That was a rhetorical question. I assume yours was too.)