Welcome to WebmasterWorld Guest from 54.166.152.121

Forum Moderators: incrediBILL

Message Too Old, No Replies

Facing crawl errors with showing missing category. How to Fix?

     
6:42 pm on Mar 23, 2014 (gmt 0)



Hello All,

I am working on a WordPress site and facing crawl errors at GWT which is showing URLs of my site without category and went on not found 404 pages.

Example is

GWT crawl error URL

www.xyz.com/world-market-center-social-media-information-sheet/

Actual URL is

www.xyz.com/socialmedia/world-market-center-social-media-information-sheet/

So most of my URLs at GWT showing with missing category that cause of crawl error.

How can i fix this please help.
6:53 pm on Mar 23, 2014 (gmt 0)

WebmasterWorld Administrator 5+ Year Member Top Contributors Of The Month



Look at your sitemaps and see that you aren't listing the same content different ways. You don't want G to crawl the same content as a tag, archive, category and page or post. Pick one format and noindex the others. It can help if you block crawling (Disallow in robots.txt) for the URLs you don't want indexed. If you need a plugin for it, I suggest YoustSEO but there are others.
8:27 pm on Mar 23, 2014 (gmt 0)

WebmasterWorld Senior Member lucy24 is a WebmasterWorld Top Contributor of All Time Top Contributors Of The Month



Does google give any hints about how it learned about the nonexistent URL? Sometimes it just makes things up. But other times you'll find it listed as either "in sitemap" (meaning some sitemap within the present geological era, not necessarily your current one) or some referring page.

If there are lots of requests and you can't pinpoint the source, it may be necessary to edit your htaccess file to put in some manual redirects. (Or, if nobody but google is making the requests, serve a 410 to make them go away.) This is Not Fun when you're working with a cms that has its own ideas about what belongs in the htaccess file.
 

Featured Threads

Hot Threads This Week

Hot Threads This Month