AV Question on spidering

[willtell]

AV has been spidering our site quite a bit. The site has about 20k pages of which all except 2k are 'noindex,follow'.

I have a few questions.

Since we don't want most pages indexed, should we make them 'noindex,nofollow' or is there some value to having the SE's go through them?

Since AV is spidering our site heavily, does that mean that they are going to index all the pages that are allowed?

[agerhart]

If you don't want AV to index some of your pages, you should block their robot, Scooter, from those pages with your robots.txt.

You could use the noindex, nofollow tag, but I think the robots.txt is a safer and more insured way to go.

[willtell]

I am looking for the 'value' of having them spider pages that we prevent from indexing by using the 'noindex' clause.