Forum Moderators: Robert Charlton & goodroi

Message Too Old, No Replies

How to increase indexing of rapidly changing website?

         

CheeryFox

10:37 am on Nov 28, 2010 (gmt 0)

10+ Year Member



I have website with events catalog. Each event has a seperate page. As event passed, page is deleted. Daily update results in disappering of hundred pages and appearing of hundred new pages. In WMT I always see thousands of 'page not found' crawl errors, because crawler tries to fetch pages which already have been deleted. How can I spend my Google crawling budget more effectively? How can I direct crawler to live pages, not to deleted ones? I do link building. The site has daily updated Sitemap. What else can be done?

aakk9999

12:45 pm on Nov 28, 2010 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



Apart from crawl budget problem you mentioned above, if any of the deleted pages got links, you lost that link equity.

Have you thought of publishing a new event on the existing page where possible, by just replacing page content?

Planet13

9:00 pm on Nov 28, 2010 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



@CheeryFox

This sounds like an interesting dilemma. Can I ask if you can clarify how your site is structured? what would a typical URL look like? (of course, you would use example addresses here)

What would the navigation structure be? Do people navigate by date (such as month)? Or do they navigate by type of event? Or do they navigate by location?

Maybe it would be best if the individual event pages used a canonical tag that linked to the immediate category with which they are associated?

Would love to hear what others think.

zefroth

8:34 am on Nov 29, 2010 (gmt 0)

10+ Year Member



Instead of deleting the outdated event pages, just label them as expired. That way you don't lose the link equity & avoid the 404s. You could also place a canonical tag on the expired pages pointing to the corresponding archive page.

You could ensure a better crawling frequency by setting up a XML Sitemap that updates dynamically as & when a new event page is added.