Welcome to WebmasterWorld Guest from 126.96.36.199
Forum Moderators: mack
The reason i'm asking is I plan to use php/mysql to create pages. My idea is to have a basic, text only html page sitting on the directory that a bot can read. I will have a index.php that will load these basic pages into a template, striping out what is not needed in order to display to a viewer. The basic page will be used to provide a "text only" version as well and will contain the keywords and other meta info.
The only way for spiders to find your pages is to follow the links. Most spiders get the home page and then move from there.
Following this (and sometimes seperate), they run it through a spam penalty/ban filter to get rid of people who are obviously trying to cheat...but that's another story :)