Forum Moderators: open
Example:
Assume that the following 2 URLs link to the same page:
1. [siteexample.com...]
2.
[siteexample.com...]
The links are very similar. The difference is link 2 contains query
string parameters (?searchId=100006648&offset=0) that are needed for
human user functionality.
The goal is to always present the same link for the same content for a robot user while not compromising the human user's experience.
POTENTIAL SOLUTION (Will this work?)
Potentially this problem can be solved by statically writing link 1 into the HTML of the page. This will allow robot users to follow link 1 to the page without any problems.
A human user that clicks on link 1 in their browser will have the link rewritten using Javascript, then followed. This satisfies the human user functionality requirements. In theory, robots are not able to "find" this link and follow it.
QUESTIONS
Which robots have built-in Javascript engines, if any?
If a robot does have a Javascript engine then how much processing will it typically perform to determine a link? Is the engine sophisticated enough to read constants, perform calculations (such as string concatenations), and run functions?
Which robots accept and use cookies, if any?
As far as I know search engines dont do anything smart with the javascript but may parse it for link like text, external js is less likely to be parsed, also dont store url as single literal. I dont know of them excepting cookies, but seems easy thing for them to do.