Forum Moderators: phranque

Message Too Old, No Replies

testing user agents & crawlers on a 404

         

compooter

2:34 am on Jan 21, 2004 (gmt 0)

10+ Year Member



Hello...

I am taking to heart ALA's article on a custom 404 page [alistapart.com], rewriting it in PHP, and testing it out.

However, I am trying to see it's results if the referring site with the invalid url is google/yahoo/msn/etc. I cannot seem to find a way to "emulate" a bad link from those sites. Is there a way to fake it? (no I'm not a spammer in disguise and I'm sorry if this sounds like a stupid idea to some)

I am writing a script to check if the referring site was a search engine, pull the previous query from that referer url, and dynamically suggest possible pages that could be what the user was really looking for. I just can't see if it works unless it really comes from a search engine.

Any ideas?

BohrMe

3:34 am on Jan 21, 2004 (gmt 0)

10+ Year Member



Have you tried wannabrowser.com [wannabrowser.com]?

compooter

4:12 am on Jan 21, 2004 (gmt 0)

10+ Year Member



Yeah - but that is geared mainly toward malicious bots. I'm trying to work with search engine referrers.

compooter

4:14 am on Jan 21, 2004 (gmt 0)

10+ Year Member



Hmm... actually I didn't realize you could type in a referrer manually - maybe it'll work after all!

hakre

6:51 pm on Jan 21, 2004 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



hey compooter,

you're looking for a tool which can do requests, shows the headers returned by the server and you can modify the referer within?

i know one quite well which does this all and it's freeware, about ~150k only (win32 only, too) and called webcow [subfiles.net].

you can just use the info-dialog for these kind of actions: copy the url of the page you'd like to test; paste it into the list; double-click it and the info-dialog appears.

modify the referer as needed and press the get info button. that's all.