Background 1. Development site was indexed by accident before launch causing numerous indexing issues (see old thread on this - [
webmasterworld.com...] )
2. I spent the whole of July cleaning up the mess, undertaking all the 301s, 410s and working with the R+D department on the CMS which was creating funky URLs
Situation Now 1. Site now has a clean index minus 2 or 3 URLs. 95% of the real HTML URLs (around 130) are indexed
2. No of pages crawled per day over the last 90 days is 465, but this is inflated by some deep crawls when I requested indexing of pages. Real average is around 200.
- The average pages crawled per day of the development/staging site was 20
- But a PPC site in my company with minimal SEO attention gets 600 pages crawled per day, (its a 2 year old site)
3. Organic traffic is minimal but i'm operating in a very competitive affiliate niche and its only been 2 months since launch
Questions 1. Is the pages crawled per day low? And thus crawl budget low? (there not being any crawl rate limit issues now)
2. To what extent is the indexing of the development site, i.e. a site which wasn't launch ready, with numerous 'bad' URLs, and which was kind of static, dead for a couple of month impeding the site's progress now?
(From Google's perspective there was this poor site online for 2-3 months where nothing was happening, it would come and do a thin crawl every day, see nothing is changed and leave and then in July there's suddenly all this on and off site activity) [edited by: goodroi at 11:55 am (utc) on Sep 11, 2017]
[edit reason] fixed url [/edit]
