frames and robots file

[betcat]

Hi

One of my clients has a site built entirely in frames. I have noticed that single frames are being indexed by the search engines and not the whole frameset. The navigation on these pages will not work properly without the fameset!

They are not prepared to put the js on each page which automatically loads the frame within the frameset and they have asked us if we can use the robots file to prevent the search engines from indexing these orphaned pages

Is this just a matter of going through their site and making a list of all of the folders and sub folders which have framed pages in them and disallowing all robots to index these pages?

ie
#
User-agent:robot_name1
Disallow:/folder1/
Disallow:/folder2/
etc

or is there an easier/quicker way of doing this (bearing in mind their dev company will not touch the html of the site!)

If i had to do this would it affect the pages that are in different folders that i want to be spidered by the search engines?

Thanks for any help in advance

[outrun]

Off Topic But I had to say something

The navigation on these pages will not work properly without the fameset!

Its a pain going to a website from a search engine that is suppose to be inside a frame and you can navigate, you probaly should convince them to put some text links for navigation maybe down the bottom for main links of the site, and a link to near the top of the site for the main page, this will also help search engine spider the whole thing.

As I understand it, if you block search engines from the frames with navigation then how will search engines find all the pages, frames arent the best way to go but there are ways to make it usuable to both search engines and the user, but its a lot more difficult.

regards,
Mark

[Macguru]

No easier or quicker way to suggest. If you block the bots, you loose trafic. If you leave individual frames in search engines index to open alone, you loose trafic too.

If they wont touch HTML, could they move a bit on JavaScript?

Here is the best way :

FORCE MANY PAGES INTO FRAMES [webmasterworld.com] see message # 21 by tedster.

[spud01]

The way i've done it in the past, is to noindex the parts of the frame set that should be not seen in a search, i.e a menu on it's own, or a header page, and so on.

While the main index page to the frame set have a <meta name="robots" content="all"> and other meta tags.

Not sure if this is correct, but I also use the robots.txt to disallow the frames folder where the headers and footer are found and so on.