Forum Moderators: bakedjake

Message Too Old, No Replies

SE spider cycles - wasted effort?

         

StephenBauer

5:48 am on May 31, 2005 (gmt 0)

10+ Year Member



A particular (and new) search engine has been spidering my site for a bit over a month and a half now. It has hit around 35K pages, yet I do not see any listed in their SERPs. If they were not intending to include me in their SERPs, would they really be indexing me so hard?

It seems like I would make a list of "do not spider" or something if they were not going to include. Maybe they have a long update window? Any thoughts?

(Which SE you ask? That pesky "Become".)

dataguy

2:48 pm on Jun 4, 2005 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



Same here... tens of thousands of pages crawled, but no listings. They are supposed to be a shopping search engine, and I don't sell anything on my sites, so I've banned their IP.