Forum Moderators: Robert Charlton & goodroi
I found an old archive with post from googleguy that listed urls spidered and asked were there more
.fcgi is faster encornation of cgi are these on your crawl list as well GoogleGuy, as it must be better if you crawl faster, and users get content served faster.
Regards Mark H
[webmasterworld.com...]
We'll crawl it all. We love cgi, asp, jsp, php, cfm, swf, ummm...
(help me out here folks; what are some other file extensions?)Let's see. html/htm/txt of course. wml? Yup, we crawl wireless markup language too, although for our wireless search. Then of course there's doc, xls, ppt, ps, ps.gz, pdf, wp (wordperfect), wri (write), tex, mdb (Access)..
Okay, I'm running out of file extensions I can think of. Maybe it would be easier to make a list of filetypes that we don't crawl? :)
[edited by: tedster at 2:50 am (utc) on Oct. 3, 2008]
[edit reason] add quote boxes [/edit]
Even though .fcgi (and .cgi for that matter) are not listed as filetypes in Google's Advanced Search and will not give any results in a filetype: operator query, nevertheless files with a .fcgi do show up in the search results.
We don't normally allow example searches here, but I'm choosing a safe and generic example - the search site:pubmedcentral.nih.gov/articlerender.fcgi returns over 400,000 results, all with a .fcgi extension.
I knew plain old perl .cgi (HTML output) I have get listed in the big G, but was not sure if I were shooting myself in the foot converting to fcgi for some new site URLs.
Thanks again!
Regards Mark H
