Welcome to WebmasterWorld Guest from 23.22.182.29

Forum Moderators: incrediBILL

Message Too Old, No Replies

Facing crawl errors with showing missing category. How to Fix?

     
6:42 pm on Mar 23, 2014 (gmt 0)

New User

joined:Feb 20, 2014
posts:4
votes: 0


Hello All,

I am working on a WordPress site and facing crawl errors at GWT which is showing URLs of my site without category and went on not found 404 pages.

Example is

GWT crawl error URL

www.xyz.com/world-market-center-social-media-information-sheet/

Actual URL is

www.xyz.com/socialmedia/world-market-center-social-media-information-sheet/

So most of my URLs at GWT showing with missing category that cause of crawl error.

How can i fix this please help.
6:53 pm on Mar 23, 2014 (gmt 0)

Moderator from US 

WebmasterWorld Administrator 5+ Year Member Top Contributors Of The Month

joined:Dec 27, 2006
posts:2563
votes: 48


Look at your sitemaps and see that you aren't listing the same content different ways. You don't want G to crawl the same content as a tag, archive, category and page or post. Pick one format and noindex the others. It can help if you block crawling (Disallow in robots.txt) for the URLs you don't want indexed. If you need a plugin for it, I suggest YoustSEO but there are others.
8:27 pm on Mar 23, 2014 (gmt 0)

Senior Member from US 

WebmasterWorld Senior Member lucy24 is a WebmasterWorld Top Contributor of All Time Top Contributors Of The Month

joined:Apr 9, 2011
posts:12704
votes: 244


Does google give any hints about how it learned about the nonexistent URL? Sometimes it just makes things up. But other times you'll find it listed as either "in sitemap" (meaning some sitemap within the present geological era, not necessarily your current one) or some referring page.

If there are lots of requests and you can't pinpoint the source, it may be necessary to edit your htaccess file to put in some manual redirects. (Or, if nobody but google is making the requests, serve a 410 to make them go away.) This is Not Fun when you're working with a cms that has its own ideas about what belongs in the htaccess file.
 

Join The Conversation

Moderators and Top Contributors

Hot Threads This Week

Featured Threads

Free SEO Tools

Hire Expert Members