Dynamic Content

Zero listed in the top 100

[startup]

Anyone else notice that Fast seems to be penalising .cgi, .shtml, .php.

[Machiavelli]

Yup

[Brett_Tabke]

I can find shtml and php pages, but stumped on finding .cgi pages. I'm surprised they even list the php and shtml pages.

[starec]

Relax, php pages are doing just fine and there is absolutely no php penalization by fast. I have tested it extensively with my pages, some of them are .php, some of them have urls generated on the fly, content nearly identical. Do it and see yourself.

[Brett_Tabke]

I was thinking the same about PHP since Fast is the only major search site to use PHP as the basis of it's corporate parent site.

[startup]

.php you say, interesting.
Dynamic content can be delivered in many forms, now I am very curious to find out why only .php.
Brett:
The shtml and php you have found, are they in the top 10 "serp" for competitve keywords?

I have found shtml, php, cgi buried deep in the results, but none in the top 100.

[littleman]

Slightly off topic semi-rant ->
It is a fools game to penalize for page extensions. The work-around to change extensions are well documented. I bet 90% of Fast's top ten are actually dynamically generated pages with htm/html extensions. The only people they are hurting are those who don't know any better.

[startup]

You got it littleman.
Just because a page is displaying a certain file extension, this does not indicate the true source. Why penalise the sites that are openly generating content. The generating of content does not give anyone an advantage because, if needs be they will use the "work around" you are referring to please the engine.
<I don't know who wrote the following>
Does this mean that we now have to make sure our hosts have the correct Apache modules installed, php and slq to please Fast.
<but I like it>

[NFFC]

Done some recent work with ASP pages, they don't seem to like those either.

[Everyman]

I haven't noticed Fast all that much, so I studied the situation. I would not say that they penalize cgi pages, as "startup" suggested in the original post. I'd say it's more like they condemn them to hell without a trial.

I have 185 static pages indexed by Fast. Very thorough job on the static pages. Some of these pages are doorways into my cgi-bin, where the dynamic pages start. Once you get going on the dynamic pages, there's no stopping.

Fast has deftly avoided any and all attempts at venturing into the cgi-bin. Just as well; at one GET per minute it would take them 83 days, and that's assuming that they're smart enough not to fetch stuff they already have. Google isn't this smart, so why would Fast be this smart?

Fast is definitely afraid of the "deep web."

[Brett_Tabke]

>condemn them to hell without a trial.

There are a few in the db. Just trying to figure out what it takes to get them in there.

[agerhart]

They don't like ASP pages at all, and about 75% of the sites I have been doing have used ASP (against my opinion and screaming). I don't even bother submitting them most of the time, I usually just create doorways and things to replace the ASP pages.

[Everyman]

Holy cow, on second thought, I take back my comment that Fast is otherwise thorough. I cannot find any of my *.txt documents in Fast!

This special directory of 134 *.txt documents is one of the most popular on my site. It's rich research from the 1980s. They aren't deep into my site -- they're at www.domain.org/gw/*.txt

None are in Fast! And when I do Fast searches for things that bring in lots of hits, like "faq" with "txt", I don't see any *.txt files coming back in the links.

Those 134 docs have been posted since early 1998, unchanged.

In June, 2000, I freaked out for a while because Google was ignoring stuff that did not have any extension at all. I was recommending to someone on another site that they change their filenames to *.txt so that Google would be happy. Well, it didn't matter because Google suddenly got much less fussy last summer.

Now it appears that even a proper *.txt extension is not cool for Fast. I just checked my friend's log, and Fast is happily crawling plain-text files with no extension at all (my friend wisely ignored my advice of last June!). Is this a bug in Fast? Perhaps by fetching but not indexing "robots.txt" they get confused and throw out all *.txt files?

Am I doing something wrong? Should I have stayed in bed this morning?

[jimmykav]

My site has lots of asp pages and a lot of them are in the db....

[agerhart]

I do not doubt that you have many ASP pages in the DB, but do any of them have a ? in the URL, or any other character like that?

It seems, or I should say it has been proven and documented, that many of the SE's won't even touch a page that has a character like this in the URL.

[agerhart]

that is to say, without hiding it.........what method do you use to account for your good rankings?