spider results

used webmasterworld spider thingy

[mattlamb]

when I spider my site using,http://www.searchengineworld.com/cgi-bin/sim_spider.cgi

I get results for my pages like this /pages/wholesale/votive.html
but other sites get :
[xxxxx.com...]

when I check links on website via holding mouse over links it reports the [etc...]

whats wrong with my website ?

[tedster]

mattlamb, I'm not clear about the question. Many browsers will convert a relative URL to a full URL onMouseover.

If that doesn't answer your question, please try rephrasing. Thanks.

[korkus2000]

You are using reletive links instead of absolute.

http:// is being added in front of your links.

Change your links from pages/wholesale/votive.html to [xxxxx.com...] in your html. Absolute links are better for search engines.

doh! tedster beat me

(edited by: korkus2000 at 1:56 am (utc) on May 15, 2002)

[DrDoc]

mattlamb,

The only thing I can think of is if you're surfing from the server .. (meaning, you have the webpages on the computer you're testing from) .. 'cause then you'd get the local URL instead.

[brotherhood of LAN]

yep its the relative URLs, I remember the same happened with me

I wouldnt change them on your site though, save yourself the extra code bloat, unless there are any drawbacks with relative URL's I dont know about.

[tedster]

I knew this dicsussion was familiar. See this thread:

Spidering Links [webmasterworld.com]

Tall Troll:
Spidering www.domain.co.uk gets the link wrong
Spidering www.domain.co.uk/ gets the link right

I bet SimSpider constructs relative paths from the entered URL (not unreasonably), by just tagging the filename called on the end of the base URL without checking for a terminal /. Also, entering .../index.html gets it right, so I bet its smart enough to sub the filename correctly, cos it'll have a / there.

So, my fault for being sloppy <grin> Who'd have guessed that?

Brett_Tabke:
Exactly. I'm using a stock module to extract links. It works on whatever it is fed.

[mattlamb]

I must be missing something, if you link from an inside page to say your home page using [.xxxxxx.com...] (absolute links) I thought the search engines saw this as spam, ie linking to yourself ?

[tedster]

No, you've had a misunderstanding, matt. That's totally legitimate linking.

I'm guessing you picked up the idea reading about people with more than one domain. Cross-linking all their domains to each other is something that some people feel gets you into trouble with Google. That's not at all the same as linking internal pages to the same domain's home page using an absolute URL - that's a pretty good practice.

The problems come about with SimSpider and trailing slashes. But absolute links are completely allowable, and often very useful. Besides being spider-friendly. it makes stealing your code a more complicated job.

[mattlamb]

Ahhhh thanks all ...I am enlightened :)