Spider just wouldn't go in 1 link in root directory

[Raymond]

I am using Brett's spider to test my website. At first the spider wouldn't go into inner pages of the website. After reading through some of the threads here, I made a sitemap page that links to most of the important pages. I placed the sitemap link on the footer of my website. It turned out the spider doesn't want to go into the sitemap page. The spider did pick up the link to the sitemap, but it did not go deeper in to the page.

FYI, it is a dynamic site that I am working on. I have already made sure there are less than 2 variables being passed between all pages. The sitemap page does not even have any variables at all and it is placed in the main root directory. The link is simply like this: sitemap.asp

Any idea?

[DaveAtIFG]

Welcome to WebmasterWorld Raymond! :)

I'm gonna offer a GUESS that the spider may have some limitations. Brett will need to comment since he wrote the durn thing.

Until I can scare him up, if you have access to your site logs, it may be useful to do some digging. See if other spiders can get to your inner pages. Knowing that may help determine if the problem is at your site/host or at Brett's spider.