Welcome to WebmasterWorld Guest from 54.167.102.69

Message Too Old, No Replies

Can I drop multiple URLs from Google's index using a wildcard?

     
6:28 am on Jun 6, 2013 (gmt 0)

New User

5+ Year Member

joined:Feb 16, 2010
posts: 9
votes: 0


I made a mistake with one of my website and a whole bunch of data got picked up by Google before I caught it.

I need to fix this relatively fast by asking Google to drop the index pages.

The URL would be something like:

domain.com/index.html?action=Abcd

Does anyone know if I can submit:

domain.com/index.html?action=*

as a single request through webmaster tools? If not, what are my other alternatives?
8:31 am on June 6, 2013 (gmt 0)

Senior Member from US 

WebmasterWorld Senior Member lucy24 is a WebmasterWorld Top Contributor of All Time Top Contributors Of The Month

joined:Apr 9, 2011
posts:12702
votes: 244


What exactly do you need to get rid of?

page name
parameter name
some value of parameter

?
9:45 am on June 6, 2013 (gmt 0)

New User

5+ Year Member

joined:Feb 16, 2010
posts: 9
votes: 0


I want to deindex all parameter urls which are more than 1 million

I want to do very carefully which doesn't impact on site home page domain.com/index.html

only kill parameter urls through webmasters tool?
12:59 pm on June 6, 2013 (gmt 0)

Junior Member

joined:Oct 24, 2011
posts: 70
votes: 0


Have you read over this page? [support.google.com...]
6:41 pm on June 6, 2013 (gmt 0)

Senior Member from US 

WebmasterWorld Senior Member lucy24 is a WebmasterWorld Top Contributor of All Time Top Contributors Of The Month

joined:Apr 9, 2011
posts:12702
votes: 244


ALL parameters? Go to the "parameters" section of wmt and tell google to ignore the parameters. They will probably already be listed.

Unless those same parameters do have meaning on other pages. Then you have to proceed more carefully.
6:52 pm on June 6, 2013 (gmt 0)

Senior Member

WebmasterWorld Senior Member Top Contributors Of The Month

joined:Sept 14, 2011
posts:730
votes: 18


You can indeed use a wild card in robots.txt something like you describe could be tested in wmt. But I would suggest to drop all parameters...

user-agent: *
disallow: /*?
3:53 am on June 7, 2013 (gmt 0)

New User

5+ Year Member

joined:Feb 16, 2010
posts: 9
votes: 0


Thanks Lucy & Jakebohall I have already set (No URLs) in URL parameters settings.

Now my question is:
(No URL) setting means Google won't crawl these urls anymore but will it deindex also? or I have to use url removal tool to drop these urls from google?

If i have to remove from url removal tool what url should I provide and will it impact my home page as well?

domain.com/index.html?action=*
5:27 am on June 7, 2013 (gmt 0)

Senior Member from US 

WebmasterWorld Senior Member lucy24 is a WebmasterWorld Top Contributor of All Time Top Contributors Of The Month

joined:Apr 9, 2011
posts:12702
votes: 244


(No URL) setting means Google won't crawl these urls anymore but will it deindex also?

Other way around. It may still crawl them if it finds them-- but results will be indexed together with the parameterless version.
5:56 am on June 7, 2013 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:Apr 5, 2002
posts:872
votes: 2


Go to the "parameters" section of wmt and tell google to ignore the parameters

From my experience, using Parameters handling at WMT isn't much of use if you want to ensure Google doesn't count those URLs that carry specific parameters. It is just a directive, and Google may choose to act on its own volition. More often that not, I have seen Google neglecting the directive given.

You can indeed use a wild card in robots.txt something like you describe could be tested in wmt

It will only help in avoiding new URLs getting indexed, but the ones already in the index, will continue to be in the index.

tkchinese, as far as I know, finding a way to add robots noindex tag is the only workable solution, if that can be managed. Else, try your luck with parameters.
7:41 am on June 7, 2013 (gmt 0)

Senior Member from US 

WebmasterWorld Senior Member lucy24 is a WebmasterWorld Top Contributor of All Time Top Contributors Of The Month

joined:Apr 9, 2011
posts:12702
votes: 244


It will only help in avoiding new URLs getting indexed

Not even that: It will only help in avoiding new URLs getting crawled. It took me at least a year to wrap my brain around this fact, so I'm not letting go of it.
10:23 am on June 7, 2013 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:Apr 5, 2002
posts:872
votes: 2


It will only help in avoiding new URLs getting crawled


:-) I will give it to you.
10:31 am on June 7, 2013 (gmt 0)

New User

5+ Year Member

joined:Feb 16, 2010
posts: 9
votes: 0


Friends I'm going to apply NoIndex...I didn't find any other solution.