How to speed up Googlebot crawl process

[serenoo]

I made some 301 redirects on the same website. But after one week if I type site:www.example.com
both pages (original and redirect) appears. On 05 Sep 2014 I submited the redirect page to google webmaster tool to the "fetch as google" function, thinking that could help, but still today I still have a duplicate content in the google index.
Is there a way to speed up that crawl process?

[phranque]

if you used fetch as googlebot and then your URL has already been crawled.
you are waiting on the indexing process. not the crawl.

[serenoo]

So what should I do to get my goal?

[rish3]

The old url may hang around forever...Google treats 301's differently these days:

See this thread: [webmasterworld.com...]

Personally, I think it's silly. A website owner should be able to decide which url is right.

[krishseo]

Check xml sitemap once, if you find the old url, then remove it from sitemap OR generate new XML sitemap and then replace with existing one and submit in webmaster tools, after that go for "Fetch as google". I this this will help you

[serenoo]

I removed the sitemap a few days ago. I read the post linked by rish3 and I am trying to put nofollow on the old page and remove it by google webmaster tool. My website is new so I will not loose anything.

[aakk9999]

How many URLs are we talking about?
The old (redirected) URLs - how far away (click path) were they from the home page?

In general, what you are experiencing is normal. Google will pick up very quickly your new URLs, but will need some time to drop old ones. If you are redirecting larger number of URLs or if URLs that you are redirecting are far away (in the click path) from the home page and do not have many links, then it can take Google some time (some months) to drop them from its index.

And in some cases, never (if Google decides that the URL you are redirecting is shorter and nicer for visitors). This usually happens with URLs that have product ID in them + some keywords not necessary for fetching the content, or where home page redirects to an internal page.

I think one week is too short to expect old URLs being dropped.

[serenoo]

We are talking about 15 URLs. The new page is www.example.com/red/01/ the old page I want to remove is www.example.com/red-01/

Anyway now I am using 404 not found. At the beginning my webmaster said it could not use it (we are developping with a CMS and the two pages are the same) and now fortunatly he found the solution so now I am able to force Google to remove www.example.com/red-01

[aakk9999]

Have you got any links pointing to the old URL www.example.com/red-01/ ? If you have, I think you should redirect or use canonical link element rather than servinig 404.

If there are no external links and you want Google to remove the page from its index as soon as possible, then it would be better to serve 410 Gone rather than 404 Not Found.

But I am wondering why are you in such a rush to get the old page out of index? If the page is clicked upon, it will redirect to the new URL anyway and Google will drop it eventually (yes, there are some edge cases when not, but you need much longer than 1 week to realise this is one of these cases).

As I said, one week is a very short time. I still do not know how far these pages are from home page (how many clicks, not how deep the folder structure in URL is), because further away they are, it generally takes longer for Google to re-crawls the page, process 301 and drops the page from its index.

On occassions I saw Google crawling the same page few times before it decides to drop the page that redirects from its index and replace it with a target page.

[serenoo]

I am in such rush because CSM create pages as they want with duplicate content and my webmaster created initially the pages without following my directions. It is not his fault because he wanted to test the template, but that pages have never been removed by Google.
I created a canonical page too, but after one week it is still there, so I decided to serve 404 because I am able to remove it right away. I read that Google can ignore the canonical tag too, so the 404 is the best solution.
I have no links pointing to www.example.com/red-01/ cause my website is new.