Welcome to WebmasterWorld Guest from 188.8.131.52
Forum Moderators: open
Great forum here.
Are any of you aware of engines discounting pages whose content dramatically changes from one visit to the next?
What I mean is I've noticed on some of my cloaked pages that a spider will visit, and then, a few minutes later, another spider will visit (from a different IP address). I can only assume this is to compare content.
We've been selling cloaking software for about a year, and haven't had any customers complain about this, but I'm wondering if SEs are getting more tricky with detecting pages.
(edited by: rcjordan at 1:03 am (utc) on April 11, 2002)
You have a valid concern. It does happen. If you are gathering a list of the IP addresses, User Agents, etc. of the second visits, I for one would very much appreciate it if you would post them in the Search Engine Spider Identification forum [webmasterworld.com].
If the content is changing because it really is that dynamic, and therefore spiders may encounter different pages when successive spiders request the same page, you may want to feed spiders static pages, or employ a method similar to the one described above. Where a dynamic page is saved and any other spider request for the same page receives the saved copy until the timer expires.
Is it really necessary? IMO no, will it help rankings? IMO yes.
incidentally, is there a good spot for a central spider ip list? i want to compare mine to others. i have about 600 ip addrs.