Forum Moderators: Robert Charlton & goodroi

Message Too Old, No Replies

How can I index ONLY the urls which are on the sitemap file?

Putting exclusive links on the sitemap file

         

Frederic1

2:59 pm on Feb 13, 2006 (gmt 0)

10+ Year Member



Hello everybody,

Is there an option to tell Google to index ONLY the urls which are listed in the sitemap.xml file?

My problem is that some nasty guys are generating duplicate content out of my site by adding extra parameters to my files.

For exemple, the url : [mysite.com...] must be unique
But some people purposely put tons of false links like :
[mysite.com...]
[mysite.com...]
[mysite.com...]

... which causes Google to downgrade my main url [mysite.com...] from the search engine results because of the duplicates.

So what I would like to do is say on my sitemap.xml file that Google must *only* index the urls which are on it and IGNORE ALL THE OTHER URLS (those found on the web).

I'm already using the robots.txt file but it's a hell to put all of the exclusions :'(

abates

12:20 am on Feb 14, 2006 (gmt 0)

10+ Year Member



as index.php is a php file, can you add in something to check if there are any arguments and do a 301 redirect to index.php without the arguments?

Frederic1

12:47 am on Feb 14, 2006 (gmt 0)

10+ Year Member



Thanks for the reply abates :-)

That's what I'm already doing (I generate a 404 header when there are bad parameters) but considering that I have many php files on my site, I can't track all the parameters for every single script :-( It would be more simple if I could tell to index only the files which are on the sitemap file but I'm not sure it's possible, nor if it's planned for the future :(