Absolute vs. Relative URLs

Do Spiders Care?

[BrettLaCroix]

I have a large online mall.

If I use an external software program to produce a site map, that map uses absolute urls.

When a spider sees an absolute URL, does it think of it as an external link (since it is the full web address) or does it figure out that it is just an absolute address pointing to the same site it is currently spidering?

Thanks,

Brett

[littleman]

BrettLaCroix, I've been using absolute urls for a couple of years now and haven't seen any negative consequences. I am pretty sure most of the major SE spiders have the ability to determine whether the link is from the same site or an outside site. They just have to do a little regular expression work -- parsing urls and comparing strings. I started to do this because I was finding duplicate pages showing up in the SEs, ones with the www and pages without. Now I do it because I do not want to leave anything to chance, sometimes bots are very stupid.

On a side note, I've been making a few spiders to do this and that. I could tell you that from a programer's prospective it is much simpler to extract hard linked pages than relative links.