Welcome to WebmasterWorld Guest from 34.229.113.106

Forum Moderators: open

Message Too Old, No Replies

Why is Google still spidering redirected pages?

     
10:18 pm on Jul 18, 2015 (gmt 0)

Full Member

5+ Year Member Top Contributors Of The Month

joined:May 11, 2012
posts: 320
votes: 3


I've noticed when I go into Google Webmaster Tools, and see Crawl Errors some of the pages it says the dead 404 pages are "linked from" have been 301 redirected to new pages. So why is Google still crawling those pages for mistakes?

I have done to websites to test 301 redirects and it says they work.
2:40 am on July 19, 2015 (gmt 0)

Administrator from US 

WebmasterWorld Administrator not2easy is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Dec 27, 2006
posts:4343
votes: 292


Every now and then Google seems to drag out their old scrapbooks and plug them in to see who's paying attention. This is how I explain it to myself when they claim to have recently crawled pages that have not existed for years and years - and surprise! They're gone. I just mark it "Fixed" and move on to more useful things.
4:17 am on July 19, 2015 (gmt 0)

Senior Member from US 

WebmasterWorld Senior Member lucy24 is a WebmasterWorld Top Contributor of All Time 5+ Year Member Top Contributors Of The Month

joined:Apr 9, 2011
posts:15698
votes: 810


some of the pages it says the dead 404 pages are "linked from" have been 301 redirected to new pages

Could you say that again in different words please? I'm bewildered.

But as not2easy says, Google never forgets an URL. I moved sites over a year and a half ago and I'm still getting occasional search-engine requests for those old URLs. But not nearly as many as in the first weeks and months. The time to worry is when there's some unusual pattern to the requests, or an unexpected change.
10:51 pm on July 23, 2015 (gmt 0)

Full Member

5+ Year Member Top Contributors Of The Month

joined:May 11, 2012
posts: 320
votes: 3


Let's say years ago I had a shopping cart that put all of it's products on a single page called /shop and so my url was store.tld/shop but now the new cart uses a different system and so I am sending all of that old traffic to my homepage, so I am 301 redirecting store.tld/shop to store.tld! If that was the case, Google should stop crawling store.tld/shop because it no longer exists and is now the homepage, right?

So, why when I get a list of crawl errors and 404 pages not found, Google Webmaster Tools tells me that some the dead links were "linked from" store.tld/shop? Why? If store.tld/shop was moved to the homepage, why is Google still crawling it for content and backlinks? I am confused.
4:21 am on July 24, 2015 (gmt 0)

Senior Member from US 

WebmasterWorld Senior Member lucy24 is a WebmasterWorld Top Contributor of All Time 5+ Year Member Top Contributors Of The Month

joined:Apr 9, 2011
posts:15698
votes: 810


"linked from" doesn't mean "linked from the most recent version of this URL according to our current crawl". It just means "this is how we found out about the URL". You may see something similar with "in sitemap" where, again, it's not necessarily in your current sitemap, or even one that has existed at any time since 2012. It's just how they first found it.

Redirecting to the home page is always a last resort, but if the previous URL really was just one page, it could be worse. Or did you mean example.com/shop?query=long-string-of-parameters-which-are-always-different-so-there-were-really-hundreds-of-URLs ?