Crawl Structure Changes - A Big No No?

[kidder]

Only this week I have made what I thougth was an improvement in the crawl structure to one of our sites - as it turns out Google did not agree and we lost all of our traffic for internal pages pretty much over night. The way it was setup Google had more than one path to crawl the pages - it seems we messed with the one they liked :)

We have set it back to the way it was - Now we play the waiting game. Does anyone know how long for?

[tedster]

Here's a thought - changing the crawl structure of a site also changes the PR circulation. If this was the cause of your drop in traffic, I'd expect to see recovery within a week, and possibly faster.

[kidder]

Thanks Ted -- When I look at it my changes may have pushed a lot of the content to the next level down so I hope you are spot on with your comment. Seems google is a bit like water and follows the easy path to the content - I made the changes simpy becasue Google has been indexing up my search results and I did not think this was the best way to work it. Guess I was wrong, another day another Google lesson.

[trinorthlighting]

Big question, did you change urls?

[kidder]

Yes but not the content URL's - I mean the destination content pages are still the same. So the search results URL's we used a script to stop them being crawled. We new content daily so Google crawls us all the time, new pages index and rank very quickly prior to the change.. We still had the archive system in place for crawl but that just runs from the index page - what I did not consider is that the search runs from every page on the site giving the bots a chance to pickup the new content 1 level from every page..

I am only talking 3 - 4 days with the changes before things went off the rails, I just hope I can get it back as we were just starting to make some ground.

[kidder]

Ted - you were about right with your prediction of 1 week. WE are now back to where we started after reverting the changes that we think caused the problems. - Nice call.