Search Engine to index my discussion pages

[Nitin]

what do I need to do to get the search engines to index my discussion pages?All my discussion pages have .cgi extension.Search Engine spiders only spider .html pages.

Any help is appreciated.

Thanks in advance.

Best Regards,
Nitin

[Krapulator]

Use some kind of server side code/component to write the cgi pages as html pages.

[Birdman]

Also, look at the query string in the urls. They can cause your pages to be excluded from the crawling.

I believe Google will index .cgi pages.

[BlobFisk]

Hi Nitin,

I'm not 100% sure what you mean by your question. Are you looking for some cgi script to output html pages?

If you're cgi pages have some server side code on them, converting them (in the simplist fashion: an extension change) to html pages will render all that code useless.

Browsers can display cgi pages perfectly well (http://www.webmasterworld.com/active.cgi [webmasterworld.com]), so if you are worried about compatability that is not an issue (the server controls how files of type .cgi are shown to the browser).

I'm not sure if I've answered your question or not, maybe a little more detail of what you are trying to achieve will help?

[RBuzz]

Why do you think that search engines are not spidering CGI pages? Run a search for

filetype:cgi inurl:cgi

on Google and see what you find. Are your CGI pages in a different format?

[gsx]

"search engines" is not Google. There are others.

In fact, other than Google, I have only ever seen FAST index CGI. They seem to have started this recently.

You need to use .htaccess and ModReWrite, but I can never get this to work (I believe it is my server that is the problem, not me), so I have no real experience, but if you look for that section on the www.apache.org website it should help. (Assuming you are using an Apache server)

[Nitin]

Thanks for your comments.

I have checked my server log files for two years but not a single cgi page is ever indexed.Does querystring creates some problem.Does search engines find some problem to index pages like this /cgi-bin/ubb/ultimatebb.cgi .I am using Ultimate Bulletin Board.My discussion pages contain very valuable information and I want them to be indexed by search engines.

Thanks for your time