Forum Moderators: Robert Charlton & goodroi

Message Too Old, No Replies

Remove Dynamic URLs from Google Index

         

learnseo

2:14 pm on Apr 12, 2010 (gmt 0)

10+ Year Member



Hello,

We have an e-commerce website with dynamic urls: "?cid=" and "?pid=" . I have added this to the robots.txt: Disallow:/*?
and I have requested to remove the url using "/filename?cid=" and "filename?pid="

This should remove all the dynamic urls from google index right?

TheMadScientist

4:53 pm on Apr 12, 2010 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



Hi learnseo,

Welcome to WebmasterWorld!

Not necessarily (depending on what you mean by requested removal). The robots.txt directive disallows GBot from crawling the URLs, so they may be included as URL only results. If you would like to remove them from the index definitively it is better to let them be spidered and add 'noindex' (<meta name="robots" content="noindex">) to the pages, which does exactly what it says and keeps them from being added to the index, which in turn keeps them out of the results. (You can also use the X_ROBOTS header in place of noindex on the pages for the same purposes.)

If you don't already have one, to get a 'super basic' view of how the spidering and indexing process works, you might have a look at post 221 of the Google Updates and SERP Changes - April 2010 [webmasterworld.com] thread. It's the top post on page 12 at 20 posts per page. To get a more 'indepth' view (and to know more about what the change to Caffeine is) check out the next post in the thread and follow the links.

NOTE: If you used the removal tool and the robots.txt correctly, yes, it will work to remove the URLs for 6 months, but then they may (will IMO) start to re-appear after 6 months as URL only, so I usually recommend using the noindex rather than robots.txt all the time, because then once they're gone you don't have to think about them coming back in some form in 6 months, and you also get at least partial credit for any inbound links to those pages.

gn_wendy

1:24 pm on Apr 13, 2010 (gmt 0)

10+ Year Member



we have also had success with the G' webmaster tools "site config/settings/parameter handling"