Welcome to WebmasterWorld Guest from

Forum Moderators: open

Message Too Old, No Replies

270 page site has .5 million results

session ID



3:57 pm on Mar 4, 2003 (gmt 0)

10+ Year Member

I just checked what FAST has indexed for my site - it shows .5 million results... but I only have 270 pages.

The site is served up from a DB - we use a sesion ID and IP in the URL (this is not a spam technique) - I assume this is the culprit for the high number of results.

Should I be worried?


7:45 am on Mar 5, 2003 (gmt 0)

10+ Year Member

yikes, thats a lotta bandwidth being eatten up. I don't think All The Web's duplicate filter is tuned in correctly because I have noticed duplicating results a few times before.


10:09 pm on Mar 5, 2003 (gmt 0)

WebmasterWorld Administrator 10+ Year Member

JP, I'd worry a bit. All things equal FAST will see your site with a lot of dupe content, which in the long run can't be good. Make sure you provide a site map etc. with links without session id's. That way the spider can crawl your site with out the id's and probably find the actual number of pages.

Dupe-like content can get you in trouble imo.


10:14 pm on Mar 5, 2003 (gmt 0)

WebmasterWorld Senior Member heini is a WebmasterWorld Top Contributor of All Time 10+ Year Member

JP, additionally you might consider sending them a message, going through ATW.
All in all there seem to be issues with their crawlers handling dynamic content.

Featured Threads

Hot Threads This Week

Hot Threads This Month