Forum Moderators: open
if i do, will it mean in all cases the robot will only look at the text pages made for older browsers?
"should i put this
<META NAME="ROBOTS" CONTENT="NOINDEX, NOFOLLOW">
on my html page that has flash, along with the browser detection server side script?"
Remember that what is described above will be detecting well known browser user agents and spider user agents, if a spider encounters the flash page, which it will because you are detecting by user agent, then it will simply ignore the flash page.
"if i do, will it mean in all cases the robot will only look at the text pages made for older browsers?"
No it won't, unless the spider uses a user agent that you are looking for in your script, or it uses a user agent of an older browser which you are also looking for in your script. Otherwise the spider will get the flash page, and with or without the noindex tag it would be bad for getting indexed.
i'm thinking i should not write a script to detect spiders either. because i'm assuming that i have to make each spider's ip address into a variable. so what if the address changes or a new spider comes along?
do i have to start detecting based on behaviour? in this case, i better understand robots in general well.
do you think "web client programming with perl" by o'reilly is a good book for this? i just don't know what to do.
You can buy a commercial cloaking script of which there are many.
You can build static pages and submit those to the search engines.
Or you can build your own script; The book you mention, is that the "camel" book? They're usually pretty good but require some base of perl knowledge to aready be in place. Here are some resources tho help you if you want to go this route:
[webmasterworld.com...]
[cgi101.com...]
[pageresource.com...]
[ebb.org...]
Is This Book for You?
...some ways in which this book may be helpful:
...For those of you who desire a better technical understanding of the Web, this book demystifies the web protocol and the browser/server interaction.
...Through web automation, much time can be saved. Repetitive tasks, like tracking packages or stock prices, can be relegated to a web robot, leaving the user free to perform more fruitful activities...
If you understand your current web environment, you are more likely to recognize areas that can be improved. Instead of waiting for solutions to show up in the marketplace, you can take an active role in shaping the future direction of your own web technology. You can develop your own specialized solutions to fit specific problems. In today's frenzied high-tech world, knowledge isn't just power, it's money. A reasonable understanding of HTTP looks nice on the resume when you're competing for software contracts, consulting work, and jobs." (Wong, Clinton., 1997, pp. viii-ix)
thanks for your help! i'll be back.
Welcome to the forums!
Is there any reason the *entire* site must be made of flash? If there is a way to put static pages up of the same site, that is much easier than cloaking, and it makes the site available to those without flash installed or those with dog slow connections.
Just something to think about.
-G
http://www.webmasterworld.com/forum21/151.htm [webmasterworld.com]
-G