Welcome to WebmasterWorld Guest from

Forum Moderators: open

Message Too Old, No Replies

270 page site has .5 million results

session ID

3:57 pm on Mar 4, 2003 (gmt 0)

Junior Member

10+ Year Member

joined:Sept 30, 2002
votes: 0

I just checked what FAST has indexed for my site - it shows .5 million results... but I only have 270 pages.

The site is served up from a DB - we use a sesion ID and IP in the URL (this is not a spam technique) - I assume this is the culprit for the high number of results.

Should I be worried?

7:45 am on Mar 5, 2003 (gmt 0)

Junior Member

10+ Year Member

joined:Sept 22, 2002
votes: 0

yikes, thats a lotta bandwidth being eatten up. I don't think All The Web's duplicate filter is tuned in correctly because I have noticed duplicating results a few times before.
10:09 pm on Mar 5, 2003 (gmt 0)

Moderator from DK 

WebmasterWorld Administrator 10+ Year Member

joined:Oct 23, 2000
votes: 10

JP, I'd worry a bit. All things equal FAST will see your site with a lot of dupe content, which in the long run can't be good. Make sure you provide a site map etc. with links without session id's. That way the spider can crawl your site with out the id's and probably find the actual number of pages.

Dupe-like content can get you in trouble imo.

10:14 pm on Mar 5, 2003 (gmt 0)

Senior Member

WebmasterWorld Senior Member heini is a WebmasterWorld Top Contributor of All Time 10+ Year Member

joined:Jan 31, 2001
votes: 0

JP, additionally you might consider sending them a message, going through ATW.
All in all there seem to be issues with their crawlers handling dynamic content.